Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add-table refactor for clean auth #9228

Merged
merged 4 commits into from
Aug 18, 2022
Merged

add-table refactor for clean auth #9228

merged 4 commits into from
Aug 18, 2022

Conversation

apucher
Copy link
Contributor

@apucher apucher commented Aug 17, 2022

We modify the AddTable admin command to use the new-ish /tableConfigs endpoint rather than /schema and /tables to fix an undesirable auth effect. This change enables use-cases where non-admin users can self-serve table creation, assuming the table names have been whitelisted ahead of time.

Previously, the /schema endpoint wasn't able to gracefully extract the table/schema name on POST, triggering a generic (i.e. non-table-specific) CREATE request against the cluster which required admin privileges.

The label has been assigned since the change makes pinot-admin incompatible with very old versions of pinot which don't support /tableConfigs yet

@apucher apucher added incompatible Indicate PR that introduces backward incompatibility bugfix cleanup security labels Aug 17, 2022
@codecov-commenter
Copy link

codecov-commenter commented Aug 17, 2022

Codecov Report

Merging #9228 (008918e) into master (3a655d2) will decrease coverage by 41.47%.
The diff coverage is 45.58%.

❗ Current head 008918e differs from pull request most recent head 42161ca. Consider uploading reports for the commit 42161ca to get more accurate results

@@              Coverage Diff              @@
##             master    #9228       +/-   ##
=============================================
- Coverage     70.01%   28.53%   -41.48%     
+ Complexity     4762       53     -4709     
=============================================
  Files          1857     1845       -12     
  Lines         99136    98782      -354     
  Branches      15076    15040       -36     
=============================================
- Hits          69406    28185    -41221     
- Misses        24833    67890    +43057     
+ Partials       4897     2707     -2190     
Flag Coverage Δ
integration1 26.38% <29.41%> (+<0.01%) ⬆️
integration2 24.90% <45.58%> (+0.10%) ⬆️
unittests1 ?
unittests2 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...roker/requesthandler/BaseBrokerRequestHandler.java 60.14% <ø> (-11.51%) ⬇️
...ot/broker/requesthandler/BrokerRequestHandler.java 0.00% <ø> (-100.00%) ⬇️
...requesthandler/MultiStageBrokerRequestHandler.java 74.07% <ø> (ø)
...t/common/response/broker/BrokerResponseNative.java 91.12% <ø> (-7.11%) ⬇️
...sources/PinotAccessControlUserRestletResource.java 0.00% <0.00%> (-61.98%) ⬇️
...ry/rules/PinotAggregateExchangeNodeInsertRule.java 0.00% <0.00%> (-91.08%) ⬇️
...inot/query/runtime/operator/AggregateOperator.java 0.00% <ø> (-83.15%) ⬇️
...ocal/recordtransformer/ComplexTypeTransformer.java 0.00% <0.00%> (-59.14%) ⬇️
.../column/DefaultNullValueVirtualColumnProvider.java 0.00% <0.00%> (-63.16%) ⬇️
...readers/constant/ConstantMVForwardIndexReader.java 0.00% <ø> (ø)
... and 1291 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@apucher apucher force-pushed the add-table-auth-cleanup branch from 81812ea to 3613589 Compare August 17, 2022 07:22
@apucher apucher force-pushed the add-table-auth-cleanup branch from e3bd481 to 379766a Compare August 17, 2022 19:46
Copy link
Contributor

@sajjad-moradi sajjad-moradi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -100,11 +91,15 @@ public void validatePermission(Optional<String> tableNameOpt, AccessType accessT
* @param httpHeaders HTTP headers containing requester identity required by access control object
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the javadoc. It's not a table level.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

public static void validatePermission(@Nullable String tableName, AccessType accessType,
@Nullable HttpHeaders httpHeaders, @Nullable String endpointUrl, AccessControl accessControl) {
String accessTypeToEndpointMsg =
String.format("access type '%s' to the endpoint '%s' for table '%s'", accessType, endpointUrl, tableName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If tableName is null, the log message doesn't look good. Previous message was handling this case better IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@apucher apucher merged commit bf66eee into master Aug 18, 2022
@apucher apucher deleted the add-table-auth-cleanup branch August 18, 2022 17:12
@npawar
Copy link
Contributor

npawar commented Sep 30, 2022

Looks like with this PR, we've lost the ability to upload REALTIME tables using AddTable which was supported before?

@npawar
Copy link
Contributor

npawar commented Sep 30, 2022

Looks like with this PR, we've lost the ability to upload REALTIME tables using AddTable which was supported before?

An oss user stumbled upon this after upgrading to 0.11 https://apache-pinot.slack.com/archives/C011C9JHN7R/p1664525974774459

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix cleanup incompatible Indicate PR that introduces backward incompatibility security
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants