Build InfluxDB query to help set transaction costs

[tao-stones]

Problem

cost_model must calculate to the same cost for a given transaction from any validator machine, for stability of the cluster. Therefore the cost of each components in proposed Fee structure must be assigned with a static (eg hardcoded) number, similar to Compute Budget.

For consistency, transaction cost should be calculated closely following proposed fee schedule, therefore the need of a consistent method to assign/collaborate compute units to each cost elements, such as signature, write lock etc.

Method:

The raw data are metrics measuring each operation's CPU cost, mostly in us. When collecting metrics, we need to get cluster-aggregated data to ride of individual validator performance differences. It takes two steps to define compute units for each Transaction cost elements:

1. The first step is get cluster-wide compute unit to us conversion rate.

This rate essentially describes the relationship between hardcoded BPF instruction's operation budget (in compute unit) vs us, which should be the same relationship for signature verification, or built-in program. In another words, it can convert other transaction cost element from us to compute unit.

PR #19624 added few more data points, if tx_wide_compute_cap feature is turned on, to add BPF instruction's compute unit and us into InfluxDB,

Run query to get conversion_rate:

select mean(rate) as conversion_rate from (SELECT "accumulated_units"::field / "execute_us"::field as rate FROM "tds"."autogen"."per_program_timings" WHERE (time > :dashboardTime: and time < :upperDashboardTime:) AND accumulated_units::field != 0 AND pubkey::field = 'all' Fill(0))

When I ran validator (PR #19624 with tx_wide_compute_cap on) on GCE VM for 15 mins, I got conversion_rate = 15.03:

2. Collect cluster wide execute time for each cost elements, using conversion_rate to convert into computye unit

• signature cost:

select mean(sig_cost) as signature_cost from (SELECT verify_batches_pp_us_mean::field / packets_mean::field as sig_cost FROM "tds"."autogen"."sigverify_stage-total_verify_time" WHERE (time > :dashboardTime: and time < :upperDashboardTime:) AND verify_batches_pp_us_mean::field != 0); 

In the same test, the query return 933 us for each signature verification, therefore its cost = 933 us * convertion_rate = 933 us * 15 cu/us = 13,995 cu.

• Write lock cost:
  The cost of write lock is rather hard to determine. A write lock potentially reduces block producing parallelism, therefore tps.

In lieu of better baseline, we could the cost of load and store an account as cost of a write lock; A transactions cost of write locks is, therefore, number of writable accounts (excluding program) * load_n_store_account_us * conversion_rate.

Query to get load_n_store_account_us:

SELECT mean(account_us) as load_n_store_account_us FROM ( SELECT (load_us::field + store_us::field)/total_account_count::field AS account_us FROM "tds"."autogen"."replay-slot-stats" WHERE (time > :dashboardTime: and time < :upperDashboardTime:) fill(0));

• Pre-byte fee for the amount of data in each loaded account.
  The reported execute timing stats includes total_data_size in transactions, and the overall execute_us, using that to roughly estimate the number of bytes can be executed in a micro-sec.

A transaction’s cost of data is (transaction.data_len / data_size_per_us) * conversion_rate.

Query to get data_size_per_us:

SELECT mean(bytes_per_us) as data_size_per_us FROM ( SELECT total_data_size::field/execute_us::field AS bytes_per_us FROM "tds"."autogen"."replay-slot-stats" WHERE (time > :dashboardTime: and time < :upperDashboardTime:) Fill(0));

• Pre-calculated fee for built-in program

Using following queries to scan cluster for mean execute micro sec for each built-in program, using rate to convert to cu.

SELECT mean(execute_us) as secp256k1_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = 'KeccakSecp256k11111111111111111111111111111');
SELECT mean(execute_us) as feature_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = 'Feature111111111111111111111111111111111111');
SELECT mean(execute_us) as incinerator_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = '1nc1nerator11111111111111111111111111111111');
SELECT mean(execute_us) as stake_config_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = 'StakeConfig11111111111111111111111111111111');
SELECT mean(execute_us) as stake_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = 'Stake11111111111111111111111111111111111111');
SELECT mean(execute_us) as config_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field ='Config1111111111111111111111111111111111111');
SELECT mean(execute_us) as vote_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = 'Vote111111111111111111111111111111111111111');
SELECT mean(execute_us) as system_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = '11111111111111111111111111111111');
SELECT mean(execute_us) as sysvar_execute_us FROM ( SELECT "execute_us"::field / "count"::field as execute_us FROM "tds"."autogen"."per_program_timings" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND execute_us::field !=0 AND count::field != 0 AND pubkey::field = 'Sysvar1111111111111111111111111111111111111');

• BPF program costs are report to cost_model in runtime.

Notes

• Once the cost is established, eg hardcoded and released, it will need a new bank feature to change the costs

[tao-stones]

Creating this panel to visualize these stats, it should help for monitoring and troubleshooting.

[tao-stones]

My InfluxDB Queries are lousy, I hope to they at least provides an consistent method to allow easily repeat the process to get cost.
@stephenhess @jackcmay Would you mind take a look at it, maybe it can be made into a automatic tool.

[tao-stones]

While #19789 being looked at, can use log to get conversion_rate:

$ cat solana-validator-C655ertkV4BUt9Cso2HVc5vQ2qoV4MKdktXZSVUPyK5Y.log | grep 'datapoint: per_program_timings' | grep -v 'pubkey=\"all\"' | grep -v 'accumulated_units=0' | sed 's/=/ /g' | sed 's/i / /g' | awk '{r = r + $13 / $11;}END{print r / NR;}'
39.8509

[tao-stones]

Here is the dashboard for testnet:
https://metrics.solana.com:8889/sources/0/dashboards/10?refresh=Paused&lower=now%28%29%20-%2015m

[tao-stones]

already done