Protocol v1 #18
Protocol v1 #18
Conversation
| val rows2 = collectRows(blockingStub.executeTable(req2)) | ||
| rows2.size shouldBe 50 | ||
|
|
||
| // This confirms the narrower qualifier is applied on top of the already-complete cache, |
There was a problem hiding this comment.
How does the test confirms the predicate was applied "on top of the already-complete cache"?
src/main/scala/com/rawlabs/das/server/cache/iterator/ExpressionEvaluator.scala
Outdated
Show resolved
Hide resolved
src/main/scala/com/rawlabs/das/server/cache/iterator/QualSelectivityAnalyzer.scala
Show resolved
Hide resolved
src/main/scala/com/rawlabs/das/server/cache/queue/AkkaChronicleDataSource.scala
Show resolved
Hide resolved
| if (validCandidates.isEmpty) { | ||
| None | ||
| } else { | ||
| // 3) Pick the cache with the fewest `cd.quals.size` |
There was a problem hiding this comment.
All candidates are valid at that point. If picking the one that had the smallest quals.size, it's going to require more filtering. We likely want the smallest cache, and apply the minimum number of predicates on top, meaning the one with the largest number of qualifiers.
There was a problem hiding this comment.
Or is there a way to pick the smallest cache (in number of rows)? That would be a decent criteria:
A query like lineitem WHERE date > X AND status = Y and country = Z has three predicates and can be served by caches:
WHERE date > X,WHERE status = Y and country = Z
It's not obvious the one with more predicates is better. Perhaps the date predicate was returning very few rows.
| columnFiltered.flatMap { ce => | ||
| QualSelectivityAnalyzer | ||
| .differenceIfMoreSelective(ce.definition.quals, requestedQuals) | ||
| .map { diff => (ce, diff) } |
There was a problem hiding this comment.
At that level we don't know if predicates were applied by the target DAS. We can't assume applying the diff qualifiers is enough.
src/test/scala/com/rawlabs/das/server/cache/iterator/QualSelectivityAnalyzerSpec.scala
Outdated
Show resolved
Hide resolved
src/main/scala/com/rawlabs/das/server/cache/iterator/QualSelectivityAnalyzer.scala
Outdated
Show resolved
Hide resolved
* predicates being considered false when not supported (in cache filtering), * edited cache logic to accept a cache that implements the exact same predicates, * edited `MessageAdapter` related code because it appeared to be a singleton reused for all tasks, * missing filter on table name, * cleared two warning stack traces indicating bad practices or missing calls to `close`, * made server read its parameters from config, * handle new cache and max-size related message fields in `Query`, * added test framework Postgres DAS.
No description provided.