Move smart vector/table to framework

[lightmark]

Description

1. add a per-bucket limit to avoid DOS
2. remove unusable inline function as [simple_map] remove inline functions #6817
3. change big_vector constructor to be friend only, the only friend is smart vector
4. move big_vector, smart_vector, smart_table to aptos_std
5. add a file in e2e-move-test with basic gas benchmarking.

After the gas change #6816, it becomes:

huge_smart_vector_create_gas |  5084900 | 0.254 | 0.763 | 1.525
huge_smart_vector_update_gas |     2100 | 0.000 | 0.000 | 0.001
huge_smart_vector_read_gas |      400 | 0.000 | 0.000 | 0.000
huge_vector_create_gas |  4080900 | 0.204 | 0.612 | 1.224
huge_vector_update_gas |  3995700 | 0.200 | 0.599 | 1.199
huge_vector_read_gas |     2600 | 0.000 | 0.000 | 0.001
huge_smart_table_create_gas |  2043500 | 0.102 | 0.307 | 0.613
huge_smart_table_update_gas |      700 | 0.000 | 0.000 | 0.000
huge_smart_table_read_gas |      300 | 0.000 | 0.000 | 0.000
huge_table_create_gas | 50594900 | 2.530 | 7.589 | 15.178
huge_table_update_gas |    50800 | 0.003 | 0.008 | 0.015
huge_table_read_gas  |      300 | 0.000 | 0.000 | 0.000

Test Plan

unit test

[movekevin]

Did you run the gas benchmarking with the 100x reduction in Alden's PR?

[lightmark]

Did you run the gas benchmarking with the 100x reduction in Alden's PR?

It's the old one. I will update.

[lightmark]

now with the new gas schedule, they outperform vector/table for large datasets in all aspects except for the creation of huge vector. I think it is both reasonable and acceptable according to current storage fee pricing. cc @msmouse

[movekevin]

ation of huge vector. I think it is both reasonable and acceptable according to current storage fee pricing. cc @msmouse

Why is creation of huge smart vector more expensive than creating a huge vector?

And how huge is huge? Is it more than 1 KB of data total?

[lightmark]

ation of huge vector. I think it is both reasonable and acceptable according to current storage fee pricing. cc @msmouse

Why is creation of huge smart vector more expensive than creating a huge vector?

And how huge is huge? Is it more than 1 KB of data total?

Because it involves some extra logic to find the right bucket, etc. It also 100% fully utilize 1kb free quota. by huge I mean way more than 1kb, here 10000 * size_of(u64) I suppose 80kb.

[clay-aptos]

Hi @lightmark , the docs here LGTM with some tiny comments I left inline. Perhaps the Move team would be alright with me editing the source files directly sometime (rather than comment on the Markdown output).

I am also asking for your, @wrwg and the team's opinion on including these wonderful additions in the relevant release notes. They seem significant, so I have created and added a release-notes tag to see if we can capture and highlight such changes.

See this issue comment for more details:
#6373 (comment)

Thanks!

[lightmark]

@clay-aptos Hey I think it is a really good idea to include it in the release notes as we wanna promoting the usage of the new gas-saving data structures.

[bowenyang007]

i trust you

[msmouse]

After the gas change #6816, it becomes

Where is the "before" numbers? 😁

[github-actions]

✅ Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> f96929550d2667d2f8eea7abf6e6e058bb347200

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> f96929550d2667d2f8eea7abf6e6e058bb347200 (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 7994 TPS, 4820 ms latency, 7000 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: f96929550d2667d2f8eea7abf6e6e058bb347200
compatibility::simple-validator-upgrade::single-validator-upgrade : 4577 TPS, 8686 ms latency, 12200 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: f96929550d2667d2f8eea7abf6e6e058bb347200
compatibility::simple-validator-upgrade::half-validator-upgrade : 4780 TPS, 8216 ms latency, 11200 ms p99 latency,no expired txns
4. upgrading second batch to new version: f96929550d2667d2f8eea7abf6e6e058bb347200
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7128 TPS, 5718 ms latency, 10100 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> f96929550d2667d2f8eea7abf6e6e058bb347200 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Pyroscope Profiling
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite land_blocking success on f96929550d2667d2f8eea7abf6e6e058bb347200

performance benchmark with full nodes : 6107 TPS, 6474 ms latency, 18500 ms p99 latency,(!) expired 560 out of 2608360 txns
Test Ok

• Grafana dashboard
• Humio Logs
• Pyroscope Profiling
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on cb4ba0a57c998c60cbab65af31a64875d2588ca5 ==> f96929550d2667d2f8eea7abf6e6e058bb347200

Compatibility test results for cb4ba0a57c998c60cbab65af31a64875d2588ca5 ==> f96929550d2667d2f8eea7abf6e6e058bb347200 (PR)
Upgrade the nodes to version: f96929550d2667d2f8eea7abf6e6e058bb347200
framework_upgrade::framework-upgrade::full-framework-upgrade : 6734 TPS, 5719 ms latency, 9700 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for cb4ba0a57c998c60cbab65af31a64875d2588ca5 ==> f96929550d2667d2f8eea7abf6e6e058bb347200 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Pyroscope Profiling
• Test runner output
• Test run is land-blocking