Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proof-producer: add trace collector tool #375

Open
wants to merge 2 commits into
base: feature/mem_expand_tests
Choose a base branch
from

Conversation

oclaw
Copy link
Contributor

@oclaw oclaw commented Mar 6, 2025

Added python script that is able to collect all the traces that are used in current circuit test set

Traces are not changed from the business logic perspective but were recollected via this new pipeline as the PoC

@oclaw oclaw force-pushed the feature/trace-collector-tool branch 2 times, most recently from 549d042 to c744a4d Compare March 6, 2025 14:33
Copy link

github-actions bot commented Mar 6, 2025

Clang Test Results

  214 files  ±    0    214 suites  ±0   40m 37s ⏱️ + 3m 59s
8 641 tests  - 1 248  8 633 ✅  - 1 248   8 💤 ±0  0 ❌ ±0 
8 888 runs   - 1 248  8 876 ✅  - 1 248  12 💤 ±0  0 ❌ ±0 

Results for commit 88423dd. ± Comparison against base commit e10e0ed.

♻️ This comment has been updated with latest results.

@oclaw oclaw force-pushed the feature/trace-collector-tool branch from c744a4d to 256443b Compare March 6, 2025 16:54
Copy link

@oclaw oclaw self-assigned this Mar 6, 2025
- "contracts/tracer_data.sol"

invocations:
- name: "simple-increment"
Copy link
Contributor Author

@oclaw oclaw Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how data preparation for the concrete test case looks like. Can be used for filtering

- name: "simple-increment"
description: "Single increment call test"
contract_name: SimpleStorage
traces_output_path: "simple/simple_inc_and_keccak"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relative path for writing output

description: "Single increment call test"
contract_name: SimpleStorage
traces_output_path: "simple/simple_inc_and_keccak"
calls:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arguments for nil_block_generator call-contract call

@oclaw oclaw force-pushed the feature/trace-collector-tool branch from 256443b to e6d1ac5 Compare March 7, 2025 10:05
@oclaw oclaw marked this pull request as ready for review March 7, 2025 10:21
@oclaw oclaw requested review from akokoshn and AndreyMlashkin March 7, 2025 10:22
Copy link

github-actions bot commented Mar 7, 2025

Gcc Test Results

  214 files  ±    0    214 suites  ±0   35m 33s ⏱️ - 2m 44s
8 641 tests  - 1 248  8 633 ✅  - 1 248   8 💤 ±0  0 ❌ ±0 
8 888 runs   - 1 248  8 876 ✅  - 1 248  12 💤 ±0  0 ❌ ±0 

Results for commit 88423dd. ± Comparison against base commit e10e0ed.

♻️ This comment has been updated with latest results.

@akokoshn akokoshn requested a review from x-mass March 8, 2025 08:10
- Define all Solitidy files you want to compile
- Prepare subdirectory for traces to be put (e.g. `my_test_case/trace_files`)
- Add invocations you want to test (contract name, method call sequence, directory for traces to be stored)
- Run `collector.py --config=collector_config.yaml --invocation=<your_invocation_name>[omit argument to recollect traces]`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be make sense add set of full command lines for update traces, or simple bash script?

Copy link
Contributor Author

@oclaw oclaw Mar 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually will consist only of mkdir + invocation of the script :) (editing of the YAML from bash does not seem reasonable to me)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it seems not make sense. As far i see collector.py --config=collector_config.yaml generate tracess for all cases.
But it's not clear from readme.
Let's replace section Trace validity test cases to Update traces with single command, and sub sections which describe how prepare data for hash miss match broken index

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to highlight this case but it obviously was not clear enough
Restructured readme, now it has sections Update traces and How to add new test case


# path to the directory with built nil (https://github.com/NilFoundation/nil) binaries
# set it to your local path
root: "/path/to/your/nil/directory"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What difference vs NIL_ROOT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIL_ROOT inside the script obtains its value from this variable, it is not an actual env var, just a placeholder for an another yaml key. Did not come with more convenient way to do this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also highlighted inside the config.yaml content btw

- Define all Solitidy files you want to compile
- Prepare subdirectory for traces to be put (e.g. `my_test_case/trace_files`)
- Add invocations you want to test (contract name, method call sequence, directory for traces to be stored)
- Run `collector.py --config=collector_config.yaml --invocation=<your_invocation_name>[omit argument to recollect traces]`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it seems not make sense. As far i see collector.py --config=collector_config.yaml generate tracess for all cases.
But it's not clear from readme.
Let's replace section Trace validity test cases to Update traces with single command, and sub sections which describe how prepare data for hash miss match broken index

@oclaw oclaw requested a review from akokoshn March 10, 2025 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants