Skip to content

Commit

Permalink
fix: minor fixes to get working (#25)
Browse files Browse the repository at this point in the history
* Closes #26
* Closes #18
* Closes #16

- completes the TODO related to meltano/sdk#1114
- Adds a default batch config because its required. Default to writing
locally.
- Add copy syntax if key properties dont exist. Previously the merge
logic was invalid due to syntax errors. Apparently my first test data
didnt have key properties
- `conform_record_data_types` requires a new `level` kwarg and was
failing because it was missing.
- the `column_exists` methods was being called a ton and was querying
snowflake for all columns every time which was slow so I have it caching
now
- the `schema_exists` method failed for me because `get_schema_names`
returns all lower columns. It errored because it was trying to create a
schema that already existed.
- bumped SDK to 0.27.0
- after bumping the SDK I needed to address
#16 which means
`create_sqlalchemy_engine` was effectively replaced with `create_engine`
- Also fix import failure of `lazy_chunked_generator` that was moved in
SDK

I'm able to test that this works with the sample tap-github data from
the getting started guide.

---------

Co-authored-by: Edgar R. M. <[email protected]>
  • Loading branch information
pnadolny13 and edgarrmondragon authored Jun 2, 2023
1 parent fbc2399 commit 6cdce06
Show file tree
Hide file tree
Showing 3 changed files with 867 additions and 800 deletions.
Loading

0 comments on commit 6cdce06

Please sign in to comment.