Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: minor fixes to get working #25

Merged
merged 10 commits into from
Jun 2, 2023
Merged

fix: minor fixes to get working #25

merged 10 commits into from
Jun 2, 2023

Conversation

pnadolny13
Copy link
Contributor

@pnadolny13 pnadolny13 commented Jun 1, 2023

  • completes the TODO related to refactor: make DDL overridable for column ADD, ALTER, and RENAME operations meltano/sdk#1114
  • Adds a default batch config because its required. Default to writing locally.
  • Add copy syntax if key properties dont exist. Previously the merge logic was invalid due to syntax errors. Apparently my first test data didnt have key properties
  • conform_record_data_types requires a new level kwarg and was failing because it was missing.
  • the column_exists methods was being called a ton and was querying snowflake for all columns every time which was slow so I have it caching now
  • the schema_exists method failed for me because get_schema_names returns all lower columns. It errored because it was trying to create a schema that already existed.
  • bumped SDK to 0.27.0
  • after bumping the SDK I needed to address chore: Upstream breaking changes to SQLConnector - please pin sdk #16 which means create_sqlalchemy_engine was effectively replaced with create_engine
  • Also fix import failure of lazy_chunked_generator that was moved in SDK

I'm able to test that this works with the sample tap-github data from the getting started guide.

@pnadolny13 pnadolny13 requested a review from kgpayne June 1, 2023 12:08
@pnadolny13 pnadolny13 changed the title bugfixes to get working fix: minor fixees to get working Jun 1, 2023
@pnadolny13 pnadolny13 marked this pull request as ready for review June 1, 2023 20:19
@pnadolny13 pnadolny13 changed the title fix: minor fixees to get working fix: minor fixes to get working Jun 1, 2023
@pnadolny13 pnadolny13 mentioned this pull request Jun 1, 2023
pyproject.toml Outdated Show resolved Hide resolved
Co-authored-by: Edgar R. M. <[email protected]>
@pnadolny13 pnadolny13 enabled auto-merge (squash) June 2, 2023 13:52
@pnadolny13 pnadolny13 merged commit 6cdce06 into main Jun 2, 2023
@pnadolny13 pnadolny13 deleted the bugfixes_to_get_working branch June 2, 2023 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
2 participants