AWS S3 Staging COPY is writing records from different table in the same raw table

[danieldiamond]

Enviroment

• Airbyte version: 0.29.11-alpha
• OS Version / Instance: AWS EC2
• Deployment: Docker
• Source Connector and version: MySQL 0.4.3
• Destination Connector and version: Snowflake 0.3.12
• Severity: Critical
• Step where error happened: Sync

Current Behavior

in the _AIRBYTE_RAW_MYTABLE i am seeing json blobs in _AIRBYTE_DATA column that are pertaining to two different tables.

i can give two examples where this happens. both of which have similar table names
e.g.

• user and user_invites tables
• entrydispute and entrydisputeevent tables

in the first table, there are JSON blobs with schemas pertaining to the second table. and so the resulting _scd and final tables contain NULL records because _AIRBYTE_RAW_USER has a record (row) with raw json data having the schema of _AIRBYTE_RAW_USER_INVITES and so the columns in this raw (in the final _scd and final table) are NULL because that row doesn't have any data with columns relating to the original table.

It appears that the AWS S3 Staging COPY is copying data from one table into the other

Expected Behavior

Each table should have _AIRBYTE_RAW_<TABLE> with only that table's data

Logs

If applicable, please upload the logs from the failing operation.
For sync jobs, you can download the full logs from the UI by going to the sync attempt page and
clicking the download logs button at the top right of the logs display window.

LOG

replace this with
your long log
output here

Steps to Reproduce

1. I am not entirely sure. I have modified connectors to use standard and then S3 staging (changing this feature back and forth) as well as CDC vs standard (changing this feature back and forth)
2. I have re-used the same source for different connections (and different tables)
3. I have made two of the same connections (one using standard and the other using S3 staging)

Are you willing to submit a PR?

Remove this with your answer.