Measure block access time
While working on another example, I've noticed that access time to some
monero blocks, especially early ones, using Blockchain::get_block_by_hash
,
is very long. Thus, I wanted check this time for all blocks in the blockchain.
As a result of this, this example was created showing how to loop through all blocks in lmdb blockchain, and extract some basic information about each block, as well as measure its access/search time, into an output csv file.
Pre-requisites
Everything here was done and tested on Ubuntu 14.04 x86_64 and Ubuntu 15.10 x86_64.
Instruction for Monero compilation:
Monero source code compilation and setup are same as here.
C++: main.cpp
int main(int ac, const char* av[]) {
// get command line options
xmreg::CmdLineOptions opts {ac, av};
auto help_opt = opts.get_option<bool>("help");
// if help was chosen, display help text and finish
if (*help_opt)
{
return 0;
}
// get other options
auto start_height_opt = opts.get_option<size_t>("start-height");
auto out_csv_file_opt = opts.get_option<string>("out-csv-file");
auto bc_path_opt = opts.get_option<string>("bc-path");
// default path to monero folder
// on linux this is /home/<username>/.bitmonero
string default_monero_dir = tools::get_default_data_dir();
// the default folder of the lmdb blockchain database
// is therefore as follows
string default_lmdb_dir = default_monero_dir + "/lmdb";
// get the program command line options, or
// some default values for quick check
size_t start_height = start_height_opt ? *start_height_opt : 0;
string out_csv_file = out_csv_file_opt ? *out_csv_file_opt : "/tmp/block_access_time.csv";
path blockchain_path = bc_path_opt ? path(*bc_path_opt) : path(default_lmdb_dir);
if (!is_directory(blockchain_path))
{
cerr << "Given path \"" << blockchain_path << "\" "
<< "is not a folder or does not exist" << " "
<< endl;
return 1;
}
blockchain_path = xmreg::remove_trailing_path_separator(blockchain_path);
cout << "Blockchain path: " << blockchain_path << endl;
// enable basic monero log output
uint32_t log_level = 0;
epee::log_space::get_set_log_detalisation_level(true, log_level);
epee::log_space::log_singletone::add_logger(LOGGER_CONSOLE, NULL, NULL);
// create instance of our MicroCore
xmreg::MicroCore mcore;
// initialize the core using the blockchain path
if (!mcore.init(blockchain_path.string()))
{
cerr << "Error accessing blockchain." << endl;
return 1;
}
// get the highlevel cryptonote::Blockchain object to interact
// with the blockchain lmdb database
cryptonote::Blockchain& core_storage = mcore.get_core();
// get the current blockchain height. Just to check
// if it reads ok.
uint64_t height = core_storage.get_current_blockchain_height();
if (start_height > height)
{
cerr << "Given height is greater than blockchain height" << endl;
return 1;
}
cout << "Current blockchain height: " << height << endl;
// output csv file
csv::ofstream csv_os {out_csv_file.c_str()};
if (!csv_os.is_open())
{
cerr << "Cant open file: " << out_csv_file << endl;
return 1;
}
cout << "Csv file: " << out_csv_file << " opened for wrting results." << endl;
// write csv header
csv_os << "Height" << "Timestamp" << "Access_time" << "Size"
<< "Hash" << "No_tx" << "Reward" << "Dificulty" << NEWLINE;
// show command line output for everth i-th block
const uint64_t EVERY_ith_BLOCK {2000};
for (uint64_t i = start_height; i < height; ++i) {
// show every nth output, just to give
// a console some break
if (i % EVERY_ith_BLOCK == 0)
cout << "Analysing block " << i << "/" << height << endl;
crypto::hash block_id;
try
{
// get block hash to be used to for the search
// in the next step
block_id = core_storage.get_block_id_by_height(i);
}
catch (const exception& e)
{
cerr << e.what() << endl;
continue;
}
cryptonote::block blk;
try
{
// measure time of accessing ith block from the blockchain
auto start = chrono::system_clock::now();
//blk = core_storage.get_db().get_block_from_height(i); // <-- alternative, faster
core_storage.get_block_by_hash(block_id, blk);
auto duration = chrono::duration_cast<chrono::nanoseconds>(chrono::system_clock::now() - start);
// get block size
size_t blk_size = core_storage.get_db().get_block_size(i);
if (i % EVERY_ith_BLOCK == 0)
cout << " - " << "access time: " << duration.count() << " ns." << endl;
// save measured data to the output csv file
csv_os << i << xmreg::timestamp_to_str(blk.timestamp)
<< duration.count() << blk_size
<< core_storage.get_block_id_by_height(i)
<< blk.tx_hashes.size()
<< cryptonote::print_money(mcore.get_block_reward(blk))
<< core_storage.get_db().get_block_difficulty(i)
<< NEWLINE;
}
catch (const exception& e)
{
cerr << e.what() << endl;
continue;
}
} // for (uint64_t i = 0; i < height; ++i)
// colose the output csv file
csv_os.flush();
csv_os.close();
cout << "\nCsv saved as: " << out_csv_file << endl;
cout << "\nEnd of program." << endl;
return 0;
}
Results
The resulted csv file can be seen here.
Screenshot of the results is here.
The plot of log of block access time against block number is here.
From the csv and data, it can be seen that there is very large concentration of very long times up to block of about 100k. This results in very slow recovery of your deterministic wallet, when the blocks in this range are being scanned for your transactions.
Compile this example
The dependencies are same as those for Monero, so I assume Monero compiles correctly. If so then to download and compile this example, the following steps can be executed:
# download the source code
git clone https://github.com/moneroexamples/block-access-time
# enter the downloaded sourced code folder
cd block-access-time
# create the makefile
cmake .
# compile
make
After this, blkaccesstime
executable file should be present in access-blockchain-in-cpp
folder. How to use it, can be seen in the above example outputs.
How can you help?
Constructive criticism, code and website edits are always good. They can be made through github.
Some Monero are also welcome:
48daf1rG3hE1Txapcsxh6WXNe9MLNKtu7W7tKTivtSoVLHErYzvdcpea2nSTgGkz66RFP4GKVAsTV14v6G3oddBTHfxP6tU