Skip to content

indexer: align affectedManifests call with matcher definitions #1514

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

crozzy
Copy link
Contributor

@crozzy crozzy commented Apr 9, 2025

This change attempts to make the matching process consistent across both the vulnerability matching and the manifest matching that is performed by the affectedManifests logic. It reduces the number of queries to 1 per matcher per vulnerability.

@crozzy crozzy added the not-review-ready This PR is currently in a state of disarray label Apr 9, 2025
@crozzy crozzy force-pushed the affected-manifests-rewrite branch 2 times, most recently from 5a04fb9 to 8e13b8b Compare April 9, 2025 17:51
@crozzy crozzy force-pushed the affected-manifests-rewrite branch from 8e13b8b to 8974b60 Compare April 9, 2025 17:55
@crozzy crozzy force-pushed the affected-manifests-rewrite branch 2 times, most recently from be269df to 1d486b5 Compare April 9, 2025 20:52
@crozzy crozzy force-pushed the affected-manifests-rewrite branch 11 times, most recently from c8d5d0b to aff1b5f Compare April 16, 2025 19:45
@crozzy crozzy removed the not-review-ready This PR is currently in a state of disarray label Apr 16, 2025
@crozzy
Copy link
Contributor Author

crozzy commented Apr 17, 2025

Execution Time not representative as it's from a small instance, but the output is still useful

Nested Loop  (cost=9.03..29.56 rows=1 width=528) (actual time=0.034..0.048 rows=4 loops=1)
  ->  Nested Loop Left Join  (cost=8.88..29.34 rows=1 width=504) (actual time=0.030..0.041 rows=4 loops=1)
        ->  Nested Loop  (cost=8.73..29.13 rows=1 width=256) (actual time=0.027..0.035 rows=4 loops=1)
              ->  Nested Loop  (cost=8.59..28.44 rows=3 width=136) (actual time=0.022..0.027 rows=6 loops=1)
                    ->  Bitmap Heap Scan on package  (cost=4.30..9.77 rows=2 width=120) (actual time=0.014..0.015 rows=2 loops=1)
                          Recheck Cond: ((name = 'pcre2'::text) AND (module = ''::text))
                          Heap Blocks: exact=1
                          ->  Bitmap Index Scan on package_unique_idx  (cost=0.00..4.29 rows=2 width=0) (actual time=0.011..0.012 rows=2 loops=1)
                                Index Cond: ((name = 'pcre2'::text) AND (module = ''::text))
                    ->  Bitmap Heap Scan on manifest_index  (cost=4.29..9.32 rows=2 width=32) (actual time=0.003..0.004 rows=3 loops=2)
                          Recheck Cond: (package_id = package.id)
                          Heap Blocks: exact=2
                          ->  Bitmap Index Scan on manifest_index_unique  (cost=0.00..4.29 rows=2 width=0) (actual time=0.001..0.001 rows=3 loops=2)
                                Index Cond: (package_id = package.id)
              ->  Index Scan using repo_pkey on repo  (cost=0.15..0.21 rows=1 width=136) (actual time=0.001..0.001 rows=1 loops=6)
                    Index Cond: (id = manifest_index.repo_id)
                    Filter: (key = 'rhel-cpe-repository'::text)
        ->  Index Scan using dist_pkey on dist  (cost=0.15..0.21 rows=1 width=264) (actual time=0.001..0.001 rows=1 loops=4)
              Index Cond: (id = manifest_index.dist_id)
  ->  Index Scan using manifest_pkey on manifest  (cost=0.15..0.22 rows=1 width=40) (actual time=0.001..0.001 rows=1 loops=4)
        Index Cond: (id = manifest_index.manifest_id)
Planning Time: 0.401 ms
Execution Time: 0.101 ms

@crozzy crozzy force-pushed the affected-manifests-rewrite branch from aff1b5f to 2821073 Compare April 17, 2025 18:30
@crozzy crozzy marked this pull request as ready for review April 17, 2025 22:02
@crozzy crozzy requested a review from a team as a code owner April 17, 2025 22:02
@crozzy crozzy requested review from vulerh and removed request for a team April 17, 2025 22:02
@crozzy crozzy force-pushed the affected-manifests-rewrite branch from 2821073 to f20042c Compare April 17, 2025 22:07
crozzy added 3 commits April 22, 2025 08:00
This change attempts to make the matching process consistent across both
the vulnerability matching and the manifest matching that is performed
by the affectedManifests logic. It reduces the number of queries to 1
per matcher per vulnerability.

Signed-off-by: crozzy <[email protected]>
Tests that do integration like behaviour should be in the root /test
directory.

Signed-off-by: crozzy <[email protected]>
Incidental import formats.

Signed-off-by: crozzy <[email protected]>
@crozzy crozzy force-pushed the affected-manifests-rewrite branch from f20042c to 6b8bff1 Compare April 22, 2025 15:01
@hdonnay
Copy link
Member

hdonnay commented Apr 25, 2025

What's the thinking on splitting the tests across two different packages?

@crozzy
Copy link
Contributor Author

crozzy commented Apr 25, 2025

What's the thinking on splitting the tests across two different packages?

The idea is that any tests that call "meta" code should be at the root level /test/ directory. These are the kind of tests (that have been know as e2e tests) which aren't directly running the code in their packages, but indirectly running it by using parent functions (i.e. calling github.com/quay/claircore/internal/matcher.Match() instead of for example using the debian matcher directly). This was already done for the rhcc package so this commit was following that pattern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants