Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FQDN and Proxy Beacon updates are not properly indexed #688

Closed
Zalgo2462 opened this issue Aug 11, 2021 · 0 comments · Fixed by #689
Closed

FQDN and Proxy Beacon updates are not properly indexed #688

Zalgo2462 opened this issue Aug 11, 2021 · 0 comments · Fixed by #689
Assignees
Labels

Comments

@Zalgo2462
Copy link
Contributor

When the FQDN analysis performs an upsert against the beaconFQDN collection, it uses the following selector:

https://github.com/activecm/rita/blob/e6c740f6ef1e2b88950eb63b1f3c6538720d0de3/pkg/beaconfqdn/repository.go#L82-L86

However, this selector is not fully covered by a MongoDB index:
https://github.com/activecm/rita/blob/e6c740f6ef1e2b88950eb63b1f3c6538720d0de3/pkg/beaconfqdn/mongodb.go#L50-L55

This means that MongoDB must select one of the indexes to perform a partial search against, and then perform a linear search over the partial search results to complete the query.

image

With some datasets, I have seen MongoDB perform a linear search over ~300 documents for each beaconFQDN upsert.

A similar problem exists in the beaconProxy code.

Selector:
https://github.com/activecm/rita/blob/e6c740f6ef1e2b88950eb63b1f3c6538720d0de3/pkg/beaconproxy/repository.go#L123-L129

Indexes:
https://github.com/activecm/rita/blob/e6c740f6ef1e2b88950eb63b1f3c6538720d0de3/pkg/beaconproxy/mongodb.go#L48-L54

Unique indexes should be added to cover these upsert selectors such that no linear searches should be performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants