Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Serialize compile request #6867

Merged
merged 16 commits into from
Feb 16, 2024
Merged

Serialize compile request #6867

merged 16 commits into from
Feb 16, 2024

Conversation

fantix
Copy link
Member

@fantix fantix commented Feb 16, 2024 •
edited
Loading

Replace QueryRequestInfo with a new serializable CompileRequest that includes:

  • Bit flags:
    • json_parameters
    • expect_one
    • inline_typeids
    • inline_typenames
    • inline_objectids
  • protocol_version
  • output_format
  • implicit_limit
  • Module aliases
  • Session config (that only affects compilation) with type descriptor
  • Serialized Source or NormalizedSource without the original query string
  • 16-byte cache key = BLAKE-2b hash of:
    • All above serialized,
    • Except that the source is replaced with Source.cache_key()
    • Plus serialized system cache that only affects compilation

@fantix
Extract allow_capabilities from QueryRequestInfo
42f5e1f 
The latter will be replaced by serializable CompileRequest
@fantix fantix force-pushed the serialized-compile-request-2 branch from a4815c0 to 42f5e1f Compare February 16, 2024 14:59
@fantix fantix force-pushed the serialized-compile-request-2 branch 2 times, most recently from f430cd5 to 218362e Compare February 16, 2024 17:52
@fantix fantix force-pushed the serialized-compile-request-2 branch from 218362e to 3ccafe4 Compare February 16, 2024 18:55
@fantix fantix requested review from elprans and msullivan February 16, 2024 18:55
@fantix fantix marked this pull request as ready for review February 16, 2024 18:55
elprans
elprans reviewed Feb 16, 2024
View reviewed changes
edb/server/compiler/compiler.py
@@ -812,6 +817,41 @@ def compute_stmt_name(text: str) -> str:

return sql_units

def compile_request(
self,
user_schema: s_schema.Schema,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be kwargs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked in #6879

msullivan
msullivan approved these changes Feb 16, 2024
View reviewed changes
Copy link
Member

@msullivan msullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Might be worth documenting in the code why we have our own serialization.

Also, almost all of my comments here are requests for more documentation. Please feel free and encouraged to ignore them for now and come back after the series of PRs is in and do them then.
(But please do actually come back and do them.)

@fantix fantix merged commit 29a1cc0 into master Feb 16, 2024
23 checks passed
@fantix fantix deleted the serialized-compile-request-2 branch February 16, 2024 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msullivan msullivan msullivan approved these changes

@elprans elprans elprans approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fantix @msullivan @elprans