Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sql-cli] Implement load file operator #1258

Merged
merged 37 commits into from
Nov 22, 2022
Merged

Conversation

pankajkoti
Copy link
Contributor

@pankajkoti pankajkoti commented Nov 16, 2022

Description

What is the current behavior?

Currently, we are not having a way for users to load data into databases.

closes: #1227

What is the new behavior?

Add load_file / LoadFileOperator to the sql-cli.

Does this introduce a breaking change?

No.

Checklist

  • Created tests which fail without the change (if possible)
  • Extended the README / documentation, if necessary

@pankajkoti pankajkoti force-pushed the 1227-sql-cli-load-operator branch from 00b59d1 to 0e27351 Compare November 16, 2022 13:26
@codecov
Copy link

codecov bot commented Nov 16, 2022

Codecov Report

Base: 94.13% // Head: 95.08% // Increases project coverage by +0.94% 🎉

Coverage data is based on head (8fc841a) compared to base (f50be8d).
Patch coverage: 97.12% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1258      +/-   ##
==========================================
+ Coverage   94.13%   95.08%   +0.94%     
==========================================
  Files          17       71      +54     
  Lines         597     3296    +2699     
  Branches       67      380     +313     
==========================================
+ Hits          562     3134    +2572     
- Misses         22      100      +78     
- Partials       13       62      +49     
Impacted Files Coverage Δ
sql-cli/sql_cli/utils/airflow.py 82.08% <ø> (+2.67%) ⬆️
sql-cli/sql_cli/utils/rich.py 88.23% <88.23%> (ø)
sql-cli/sql_cli/dag_generator.py 93.33% <94.44%> (ø)
sql-cli/sql_cli/workflow_directory_parser.py 97.84% <97.84%> (ø)
sql-cli/sql_cli/__init__.py 100.00% <100.00%> (ø)
sql-cli/sql_cli/__main__.py 99.06% <100.00%> (+1.84%) ⬆️
sql-cli/sql_cli/connections.py 100.00% <100.00%> (ø)
sql-cli/sql_cli/constants.py 100.00% <100.00%> (ø)
sql-cli/sql_cli/dag_render.py 100.00% <100.00%> (+11.11%) ⬆️
sql-cli/sql_cli/exceptions.py 100.00% <100.00%> (ø)
... and 59 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@pankajkoti pankajkoti force-pushed the 1227-sql-cli-load-operator branch from 2213b22 to 4214530 Compare November 18, 2022 07:58
@pankajkoti pankajkoti marked this pull request as ready for review November 18, 2022 10:31
@pankajkoti pankajkoti requested a review from feluelle November 18, 2022 10:31
@pankajkoti pankajkoti requested a review from tatiana November 18, 2022 13:50
@pankajkoti pankajkoti changed the title Initial commit for load operator interface [sql-cli] Implement load file operator Nov 21, 2022
@pankajkoti pankajkoti requested a review from feluelle November 21, 2022 09:33
Copy link
Member

@feluelle feluelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Let's only fix the remaining failed SQL CLI test.

All other failing tests are related to python-sdk. :(

@pankajkoti pankajkoti requested a review from feluelle November 21, 2022 11:32
@pankajkoti pankajkoti force-pushed the 1227-sql-cli-load-operator branch from 60748d3 to ce52863 Compare November 21, 2022 21:04
@pankajkoti pankajkoti requested a review from tatiana November 22, 2022 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate load file operator together with generate and run in SQL CLI
3 participants