Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache Results of the Exists CQL Expression #1051

Closed
12 of 14 tasks
juliangruendner opened this issue Jul 31, 2023 · 0 comments
Closed
12 of 14 tasks

Cache Results of the Exists CQL Expression #1051

juliangruendner opened this issue Jul 31, 2023 · 0 comments
Assignees
@alexanderkiel
Labels
enhancement New feature or request module:cql module:evaluate-measure module:frontend
Milestone
v0.28.0

Comments

@juliangruendner
Copy link
Contributor

juliangruendner commented Jul 31, 2023 •
edited by alexanderkiel
Loading

The CQL cache is deactivated per default. In order to activate it, please set the env var CQL_EXPR_CACHE_SIZE to a value of the size of the cache in MiB.

Todos

  • generate new PatientAsOf index on the fly without the need to generate all indices
    • generation works
    • investigate memory leak
    • prevent repetitive index generation on once successfully generated index
  • ensure that all CQL expressions have a form
  • test with real data and evaluate the cache hit ratio
  • allow to disable the cache
  • document the improved performance results
    • 100k on LEA47
    • 1M on LEA47
    • 1M on LEA58
  • don't cache excessive big expressions
  • constrain the Bloom filter generation queue
  • show stats in the UI
@juliangruendner juliangruendner linked a pull request Jul 31, 2023 that will close this issue
[WIP] CQL Cache #1050
Closed
@alexanderkiel alexanderkiel changed the title Extend blaze with a cql cache Extend Blaze with a CQL Cache Aug 1, 2023
alexanderkiel added a commit that referenced this issue Nov 9, 2023
@alexanderkiel
Fix Concurrent Use of KV Iterators
ae85e62 
The cost of opening an iterator doesn't impact the query performance in
a significant way. Especially if we have a CQL Cache through #1051 the
performance impact would be negligible.

Closes: #1217
@alexanderkiel alexanderkiel mentioned this issue Nov 9, 2023
Merged
@alexanderkiel alexanderkiel added this to the v0.28.0 milestone May 28, 2024
@alexanderkiel alexanderkiel changed the title Extend Blaze with a CQL Cache Cache Results of the Exists CQL Expression Jun 12, 2024
alexanderkiel added a commit that referenced this issue Jun 12, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
4d8b652 
Closes: #1051
@alexanderkiel alexanderkiel mentioned this issue Jun 12, 2024
Merged
alexanderkiel added a commit that referenced this issue Jun 12, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
ebbf881 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 12, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
6ed6a7a 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 13, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
6e7584e 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 13, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
584eec2 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 13, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
74fa3d0 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 14, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
ef4527e 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 17, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
7f06c4d 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 17, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
c3c7c08 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 18, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
41d6e82 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 23, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
867363b 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Add Record for Complex Type Ratio
6cfe6c8 
Needed in: #1051
@alexanderkiel alexanderkiel mentioned this issue Jun 24, 2024
Merged
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Add Record for Complex Type Ratio
1c597c3 
Needed in: #1051
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Add Record for Complex Type Ratio
4497165 
Needed in: #1051
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Add Estimated Number of Keys Function
994694d 
We need this functionality for all key-value instead of only for
RocksDB because we will use it in the CQL expression cache.

Needed in: #1051
@alexanderkiel alexanderkiel mentioned this issue Jun 24, 2024
Merged
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Migrate Cache Collector into it's own Module
f671c4e 
We need the cache collector in the CQL expression cache.

Needed in: #1051
@alexanderkiel alexanderkiel mentioned this issue Jun 24, 2024
Merged
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Migrate Cache Collector into it's own Module
e793ad5 
We need the cache collector in the CQL expression cache.

Needed in: #1051
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Migrate Cache Collector into it's own Module
a2d8afe 
We need the cache collector in the CQL expression cache.

Needed in: #1051
alexanderkiel added a commit that referenced this issue Jun 24, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
299ef44 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 25, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
17b07f4 
Closes: #1051
alexanderkiel added a commit that referenced this issue Jun 25, 2024
@alexanderkiel
Cache Results of the Exists CQL Expression
8717e3b 
Closes: #1051
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexanderkiel alexanderkiel

Labels
enhancement New feature or request module:cql module:evaluate-measure module:frontend
Projects
None yet
Milestone
v0.28.0
Development

Successfully merging a pull request may close this issue.

[WIP] CQL Cache samply/blaze
2 participants
@alexanderkiel @juliangruendner