Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NFT Metadata Crawler] Add column to mark URIs to not be parsed #11846

Merged
merged 1 commit into from
Feb 1, 2024

Conversation

just-in-chang
Copy link
Contributor

Description

Many submitted tokens do not have proper JSON or image NFT metadata URIs. This column flags that so that they are not reprocessed and lets us have more accurate metrics regarding the success rate of the parser.

Test Plan

Tested locally + existing unit tests

Copy link

trunk-io bot commented Jan 31, 2024

@just-in-chang just-in-chang requested a review from a team January 31, 2024 23:35
@just-in-chang just-in-chang enabled auto-merge (squash) January 31, 2024 23:46

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Feb 1, 2024

✅ Forge suite compat success on aptos-node-v1.8.3 ==> 5d1ade568a077e9a6df53b55ef7a4f7b9fdeb78d

Compatibility test results for aptos-node-v1.8.3 ==> 5d1ade568a077e9a6df53b55ef7a4f7b9fdeb78d (PR)
1. Check liveness of validators at old version: aptos-node-v1.8.3
compatibility::simple-validator-upgrade::liveness-check : committed: 2710 txn/s, latency: 6985 ms, (p50: 6000 ms, p90: 7900 ms, p99: 23500 ms), latency samples: 181580
2. Upgrading first Validator to new version: 5d1ade568a077e9a6df53b55ef7a4f7b9fdeb78d
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1793 txn/s, latency: 16438 ms, (p50: 18600 ms, p90: 22900 ms, p99: 23200 ms), latency samples: 91480
3. Upgrading rest of first batch to new version: 5d1ade568a077e9a6df53b55ef7a4f7b9fdeb78d
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1836 txn/s, latency: 15935 ms, (p50: 18900 ms, p90: 22200 ms, p99: 22300 ms), latency samples: 93640
4. upgrading second batch to new version: 5d1ade568a077e9a6df53b55ef7a4f7b9fdeb78d
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3646 txn/s, latency: 8807 ms, (p50: 9900 ms, p90: 12000 ms, p99: 12600 ms), latency samples: 142220
5. check swarm health
Compatibility test for aptos-node-v1.8.3 ==> 5d1ade568a077e9a6df53b55ef7a4f7b9fdeb78d passed
Test Ok

Copy link
Contributor

github-actions bot commented Feb 1, 2024

✅ Forge suite realistic_env_max_load success on 5d1ade568a077e9a6df53b55ef7a4f7b9fdeb78d

two traffics test: inner traffic : committed: 6008 txn/s, latency: 6332 ms, (p50: 5700 ms, p90: 8800 ms, p99: 16300 ms), latency samples: 2613600
two traffics test : committed: 100 txn/s, latency: 2763 ms, (p50: 2300 ms, p90: 3000 ms, p99: 14500 ms), latency samples: 1880
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.221, avg: 0.197", "QsPosToProposal: max: 0.178, avg: 0.169", "ConsensusProposalToOrdered: max: 0.630, avg: 0.615", "ConsensusOrderedToCommit: max: 0.557, avg: 0.508", "ConsensusProposalToCommit: max: 1.171, avg: 1.123"]
Max round gap was 1 [limit 4] at version 662201. Max no progress secs was 8.614764 [limit 15] at version 2578036.
Test Ok

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add if exists and if not exists to make sure that the migration doesn't fail.

@just-in-chang just-in-chang merged commit 8d7d6f0 into main Feb 1, 2024
79 of 80 checks passed
@just-in-chang just-in-chang deleted the justin/do-not-parse branch February 1, 2024 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants