-
Notifications
You must be signed in to change notification settings - Fork 14.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Iceberg provider #39155
Add Iceberg provider #39155
Conversation
@Fokko Do we want to keep Tabular provider (if there might be future tabular specific behavior?) or should we deprecate the whole tabular provider in favor of apache-iceberg provider? (difference between deprecate specific classes vs the whole provider) |
No, let's deprecate it in favor of the Iceberg provider. Everything in here is just Iceberg, and nothing Tabular specific, so the current naming is confusion |
@romsharon98 Sorry for being late to the party here, but this looks great! Thanks for picking this up 🎉 |
e8b8d3e
to
4726ce3
Compare
45366db
to
c06823b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some small comments, but this looks great @romsharon98
c06823b
to
9710a4e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Marking as request changes just to wait for my review and after release of current provider wave
I'll get to it in 1-2 days
@eladkal That makes sense. It was not my intent to merge this without your approval, but just to indicate that there are no further comments on the Operator and Hook itself. |
fc0a936
to
b352f39
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
will merge when CI is green
3fa715a
to
a81d839
Compare
🎉🎉🎉 |
Wooohooo! |
Nice! Thanks @romsharon98 for working on this, and thanks @eladkal and @potiuk for the review 🎉 |
When the tabular provider was deprecated in favor of iceberg in apache#39155, it appears there was a bad merge conflict resolution that wiped out the existence of 1.5.0.
* add log for running callback * revert * add iceberg provider * add tabular deprecation * fix comment * fix comment * fix redirect * deprecated tabular, fix integration name * fix connections.rst * merge with main * remove iceberg default host * add to providers bug report * fix changelog and revert __init__ * remove redirects, remove tabular new version, revert latest docs only * remove deperecated hook * add deprecation warning * revert tabular connection and add iceberg connection * fix iceberg tests * fix iceberg connection test * fix iceberg connection * mock the correct connection * tabular should not have tests * remove deprecated hook from yaml * fix integration name * add iceberg logo * fix integrations * fix iceberg logo location * revert tabular in extra-packages-ref * fix docs
* add log for running callback * revert * add iceberg provider * add tabular deprecation * fix comment * fix comment * fix redirect * deprecated tabular, fix integration name * fix connections.rst * merge with main * remove iceberg default host * add to providers bug report * fix changelog and revert __init__ * remove redirects, remove tabular new version, revert latest docs only * remove deperecated hook * add deprecation warning * revert tabular connection and add iceberg connection * fix iceberg tests * fix iceberg connection test * fix iceberg connection * mock the correct connection * tabular should not have tests * remove deprecated hook from yaml * fix integration name * add iceberg logo * fix integrations * fix iceberg logo location * revert tabular in extra-packages-ref * fix docs
Following #32786 (comment) since Tabular provider doesn't have any specific tabular logic it has been suggested to move Tabular logic to new Apache Iceberg provider
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.