Add subcommand "compute-slot-cost" to ledger tool

[tao-stones]

Problem

Want a way to run banking-stage's cost-model over the block of given slot(s), to compute how expensive each block was, which is used as input for tuning the cost_model that banking_stage uses to feeds its banks.

Summary of Changes

• Add positional compute-slot-cost subcommand to ledger tool, it runs on all slot if no slots given on command line;
• It runs each slot's block through cost_model, if a block breaches the cost_model limit, a message will be printed; otherwise each block's cost metrics are printed.

Fixes #

[codecov]

Codecov Report

Merging #17181 (0d91203) into master (597373f) will increase coverage by 0.0%.
The diff coverage is 91.8%.

❗ Current head 0d91203 differs from pull request most recent head 41b19d4. Consider uploading reports for the commit 41b19d4 to get more accurate results

@@           Coverage Diff            @@
##           master   #17181    +/-   ##
========================================
  Coverage    82.7%    82.7%            
========================================
  Files         421      419     -2     
  Lines      117926   118112   +186     
========================================
+ Hits        97574    97776   +202     
+ Misses      20352    20336    -16     

[tao-stones]

Using this branch and ledger downloaded from mainnet, ran $cargo run --bin solana-ledger-tool -- -l mainnet-beta/validator-ledger compute-cost --all to compute costs for all slots' blocks. Results are at google spreadsheet: https://docs.google.com/spreadsheets/d/1bMdUyTFiNEcbTkxS500VNFh7yL0z5c0irqGQK7dmVLs/edit?usp=sharing
Where a vote instruction is considered a unit of cost.

two things are interesting (from the spreadsheet):

1. The block of slot 76927422 has largest cost (log snippet below), where almost all costs are for one writable account (can filter by slot to see the details on spreadsheet). This is the classic case in which cost_model would spread transactions on same account to multiple blocks.

% cargo run --bin solana-ledger-tool -- -l mainnet-beta/validator-ledger compute-cost --slots 76927422
... ...
Slot: 76927422, Entries: 925, Transactions: 989, Programs 1118, CostModelStats { total_cost: **575687**, number_of_accounts: 823, costliest_account: 9UJ1AYv6rko5xu4Z7DnA5hLaNZFxDu9BxqKp1LULivqC, costliest_account_cost: **559000** }
  Programs: {11111111111111111111111111111111: 120, 675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8: 10, 9xQeWvG816bUx9EPjHmaT23yvVM2ZWbrrpZb9PusVFin: 141, BPFLoaderUpgradeab1e11111111111111111111111: 559, EhhTKczWMGQt46ynNeRX1WfeagwwJd7ufHvCDjRxjo5Q: 1, Vote111111111111111111111111111111111111111: 287}

2. the account agsWhfJ5PPGjmzMieWY8BR5o1XRVszUBQ5uFz4CtDiJ appeared far more frequently than any other accounts to be the costliest account - meaning the account most instructions/programs wants to write in same block. In other words, this account blocks lots of transactions from being executed in parallel.

To improve cluster throughput efficiency with cost_model, my thought is to set block cost limit to the the max value in spreadsheet (and round up for growth) to 1,000,000 (equivalent to 1M Vote instructions), and to limit the account cost limit to something below average (say equivalent 10,000 vote instructions) to promote parallelizable transactions in a block.

@sakridge @mvines what are your thoughts?

[mvines]

lgtm overall, just some minor nits. Note that I'm only reviewing the ledger-tool/ changes in this PR, since the rest of this code is from another PR

[mvines]

Ugh, it hurts to see this dependency come into ledger-tool. Oh well, we need to refactor core/ more at some point anyway I guess

[tao-stones]

Ah, see your point, core should be internal and unexposed. Maybe make sense to move cost model from core to other better suited crates, I am not sure where though, maybe sdk or runtime?

[tao-stones]

@mvines cost_model PR has been merged to master, rebased this PR, it's ready for review.

[mvines]

looks good