Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instance-wide params should only be injected if a connector will be using oauth #8018

Merged
merged 4 commits into from
Nov 17, 2021

Conversation

ChristopheDuong
Copy link
Contributor

@ChristopheDuong ChristopheDuong commented Nov 16, 2021

What

Closes #5989

How

  • Since we removed the webbackend/sources/create endpoints for oauth and merge it into the completeOAuth ones in this PR, we don't need to inject masked values into connector configs anymore. Therefore, the OAuthConfigSupplier is only used by the Scheduler at runtime to inject real OAuth credentials.
  • The OAuthConfigSupplier injects values into the connector config by first checking oauth predicates from the OAuthConfigSpecification object attached to the connector's spec object. It also uses it to filter unwanted property outputs before merging them in the right location as specified by the field path_in_connector_config

Recommended reading order

  1. airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/job_factory/OAuthConfigSupplier.java
  2. the rest

@github-actions github-actions bot added area/platform issues related to the platform area/scheduler area/server labels Nov 16, 2021
@ChristopheDuong ChristopheDuong temporarily deployed to more-secrets November 16, 2021 16:40 Inactive
@ChristopheDuong ChristopheDuong temporarily deployed to more-secrets November 16, 2021 17:07 Inactive
@ChristopheDuong ChristopheDuong temporarily deployed to more-secrets November 16, 2021 18:20 Inactive
@ChristopheDuong ChristopheDuong temporarily deployed to more-secrets November 16, 2021 18:48 Inactive
.withCompleteOauthServerOutputSpecification(Jsons.jsonNode(Map.of(
API_CLIENT, Map.of(
"type", "string",
OAuthConfigSupplier.PATH_IN_CONNECTOR_CONFIG, List.of(CREDENTIALS, API_CLIENT)))))))));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ConnectorSpecification creation logic can be put into a helper method. This logic is needed in multiple places, and most of them share 90% of the parameters. Currently it is a bit hard to read, especially to tell the differences between difference test cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point thanks!

@ChristopheDuong ChristopheDuong temporarily deployed to more-secrets November 17, 2021 10:12 Inactive
@ChristopheDuong ChristopheDuong merged commit 2a153bc into chris/oauth-consent-url Nov 17, 2021
@ChristopheDuong ChristopheDuong deleted the chris/oauth-injection branch November 17, 2021 18:33
ChristopheDuong added a commit that referenced this pull request Nov 17, 2021
* Refactor OAuth consent flow with new API

* Use input connector configuration in getConsentURL for OAuth flow

* Instance-wide params should only be injected if a connector will be using oauth (#8018)

* Add test for nested parameters
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
…q#7983)

* Refactor OAuth consent flow with new API

* Use input connector configuration in getConsentURL for OAuth flow

* Instance-wide params should only be injected if a connector will be using oauth (airbytehq#8018)

* Add test for nested parameters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants