Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix JoinDataSource to not ignore DataSource layers silently #17726

Open
wants to merge 138 commits into
base: master
Choose a base branch
from

Conversation

kgyrtkirk
Copy link
Member

  • JoinDataSource uses DataSourceAnalysis as a model - but during construction it ignores some other typed DataSource-s
  • there was a hack to possibly make those work - but it could have been easily happened that it just forgot to call the createSegmentMapFunction on those sources
  • at the same time DataSourceAnalysis also acts as a vertex-boundary like object; which makes this a bit more complicated.

This change is limited to fix the bug: I think in a followup will be needed to create a separate object which could be used inside the join - and a separate one to work with the boundary - the DataSource#getAnalysis was being used at a lot of places so the bugfix was separated for now.

- Rename createReport() to createModesReport() to reflect the updated functionality.
- Update getAnnotation() to getModesAnnotation() to align with the new annotation type.
- Add a new test method createQuidemReasonReport() to handle DecoupledTestConfig annotations.
- Update getQuidemReasonAnnotations() to handle DecoupledTestConfig annotations.
- Update createReport() to handle both NotYetSupported and DecoupledTestConfig annotations.

This commit introduces changes to the test methods and annotations to improve clarity and consistency in the test suite.
This reverts commit 173a057.
This reverts commit 293a483.
This reverts commit c236af7.
@github-actions github-actions bot added Area - Batch Ingestion Area - Querying Area - Segment Format and Ser/De Area - Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Feb 13, 2025
}

/**
* Flatten a datasource into two parts: the left-hand side datasource (the 'base' datasource), and a list of join
* clauses, if any.
* @param b

Check notice

Code scanning / CodeQL

Spurious Javadoc @param tags

@param tag "b" does not match any actual parameter of method "flattenJoin()".
@@ -485,10 +485,11 @@
@Test
public void testJoinUnderTopLevelSubqueries()
{
final JoinDataSource joinDataSource;

Check notice

Code scanning / CodeQL

Unread local variable

Variable 'JoinDataSource joinDataSource' is never read.
@@ -107,13 +109,23 @@
return getQuerySegmentSpecForLookUp(this).lookup(this, walker);
}

public DataSourceAnalysis getDataSourceAnalysis()

Check notice

Code scanning / CodeQL

Missing Override annotation

This method overrides [Query<T>.getDataSourceAnalysis](1); it is advisable to add an Override annotation.
@kgyrtkirk kgyrtkirk marked this pull request as ready for review February 19, 2025 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area - Batch Ingestion Area - Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Area - Querying Area - Segment Format and Ser/De
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant