Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Rework core.Spans #3210

Merged
merged 5 commits into from
Nov 6, 2024

Conversation

AndrewSisley
Copy link
Contributor

Relevant issue(s)

Resolves #3209

Description

Simplifies core.Spans a little bit, before removing some old hacky code allowing a nicer way of handling headstore keys within the planner/fetcher system.

Is probably easier to review this commit by commit, the changes are documented by the commit messages.

@AndrewSisley AndrewSisley added area/query Related to the query component refactor This issue specific to or requires *notable* refactoring of existing codebases and components code quality Related to improving code quality labels Nov 5, 2024
@AndrewSisley AndrewSisley added this to the DefraDB v0.15 milestone Nov 5, 2024
@AndrewSisley AndrewSisley requested a review from a team November 5, 2024 22:15
@AndrewSisley AndrewSisley self-assigned this Nov 5, 2024
Copy link

codecov bot commented Nov 5, 2024

Codecov Report

Attention: Patch coverage is 79.78723% with 38 lines in your changes missing coverage. Please review.

Please upload report for BASE (develop@c8fd3b1). Learn more about missing BASE report.
Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
internal/keys/headstore_doc.go 68.75% 5 Missing ⚠️
internal/db/fetcher/versioned.go 73.33% 1 Missing and 3 partials ⚠️
internal/planner/commit.go 88.46% 2 Missing and 1 partial ⚠️
internal/planner/values.go 0.00% 3 Missing ⚠️
internal/keys/key.go 66.67% 2 Missing ⚠️
internal/planner/scan.go 77.78% 1 Missing and 1 partial ⚠️
internal/planner/view.go 0.00% 2 Missing ⚠️
internal/planner/arbitrary_join.go 0.00% 1 Missing ⚠️
internal/planner/average.go 80.00% 1 Missing ⚠️
internal/planner/count.go 0.00% 1 Missing ⚠️
... and 14 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             develop    #3210   +/-   ##
==========================================
  Coverage           ?   77.34%           
==========================================
  Files              ?      376           
  Lines              ?    34791           
  Branches           ?        0           
==========================================
  Hits               ?    26906           
  Misses             ?     6263           
  Partials           ?     1622           
Flag Coverage Δ
all-tests 77.34% <79.79%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/core/data.go 91.04% <100.00%> (ø)
internal/db/collection_get.go 84.48% <100.00%> (ø)
internal/db/collection_index.go 87.53% <100.00%> (ø)
internal/db/fetcher/dag.go 62.50% <100.00%> (ø)
internal/db/fetcher/errors.go 16.00% <ø> (ø)
internal/db/fetcher/fetcher.go 79.00% <100.00%> (ø)
internal/db/fetcher/indexer.go 83.92% <100.00%> (ø)
internal/keys/datastore_doc.go 82.18% <100.00%> (ø)
internal/lens/fetcher.go 70.23% <100.00%> (ø)
internal/planner/multi.go 80.30% <100.00%> (ø)
... and 27 more

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8fd3b1...6eb7987. Read the comment docs.

Comment on lines +295 to +300
"spans": []dataMap{
{
"end": "/4",
"start": "/3",
},
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: So this was the bug that was caused due to the way HasValue was being used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, there was something fishy going on here with HasValue. The newly expected result is correct, and the old property/type has died so I didn't bother chasing down exactly where the bug was

Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Praise: I like the change, specially the de-abstraction of the interface into struct and the removal of HasValue

} else {
valueSpans[i] = core.NewSpan(span.Start.WithValueFlag(), span.End.WithValueFlag())
// DocumentFetcher only ever recieves document keys
//nolint:forcetypeassert
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Have a must DataStoreKey function for the casting? and do this lint ignore there once for all these 4 casts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think moving a simple cast into a function would improve the readability of the code here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can also move span.Start.(keys.DataStoreKey) and span.End.(keys.DataStoreKey) out of if-else blocks into a variable, and then use that in their respective occurrences. Feel free to ignore as this seems like subjective pref

@@ -41,7 +41,7 @@ type DataStoreKey struct {
FieldID string
}

var _ Key = (*DataStoreKey)(nil)
var _ Walkable = (*DataStoreKey)(nil)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: What inspired the name change, and the context for the name "walkable"? I am guessing it's the key that can be accessed.

EDIT: Was answered by documentation on Walkable ignore this question.

Comment on lines +24 to +29
// Walkable represents a key in the database that can be 'walked along'
// by prefixing the end of the key.
type Walkable interface {
Key
PrefixEnd() Walkable
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Answered my question in another comment, ignore the previous question please.

@@ -142,13 +144,14 @@ func (n *dagScanNode) simpleExplain() (map[string]any, error) {

// Build the explanation of the spans attribute.
spansExplainer := []map[string]any{}
undefinedHsKey := keys.HeadStoreKey{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick:

Suggested change
undefinedHsKey := keys.HeadStoreKey{}
undefinedHSKey := keys.HeadStoreKey{}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Headstore is one word, the current capitalization of HeadStoreKey is incorrect and has been fixed in the branchable collections branch (is bundled with some other branchable-specific changes, which is why it isnt in this PR).

Is not required, and actually seems to be buggy
Type alias is temporary, I'll remove the Spans type soon
There is only one implementation, and given the consuming code structure in the planner it is highly likely that there will only ever be one Spans implementation.
This is the primary goal of the PR, as otherwise I'd have to extend the hacks even further in order to query collection commits.
@AndrewSisley AndrewSisley merged commit 6d6c9f2 into sourcenetwork:develop Nov 6, 2024
42 of 43 checks passed
@AndrewSisley AndrewSisley deleted the 3209-rework-spans branch November 6, 2024 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/query Related to the query component code quality Related to improving code quality refactor This issue specific to or requires *notable* refactoring of existing codebases and components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rework core.Spans
2 participants