Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load MSSQL source with no system tables #2260

Open
tommy1x opened this issue Feb 3, 2025 · 0 comments
Open

Load MSSQL source with no system tables #2260

tommy1x opened this issue Feb 3, 2025 · 0 comments

Comments

@tommy1x
Copy link

tommy1x commented Feb 3, 2025

DLT version 1.6.0
I'm trying to load data from a Microsoft Dataverse TDS endpoint.
I'm setting up the connection by providing a SQLAlchemy engine instance, so I can use a Microsoft Entra ID token for authentication.
I'm getting the following error:
[Microsoft][ODBC Driver 18 for SQL Server][SQL Server]{"Message":"Invalid table name COLUMNS for schema INFORMATION_SCHEMA.

The TDS endpoint seems to have sys tables (e.g. sys.tables, sys.columns) as well as INFORMATION_SCHEMA.TABLES, but it doesn't seem to have INFORMATION_SCHEMA.COLUMNS.
I can get around it by specifying metadata for the load but that isn't ideal. Is there any way to load it without it trying to query INFORMATION_SCHEMA.COLUMNS, even without provided metadata?

This is roughly the code I've tried which works when providing metadata:

metadata = MetaData()
Table(
    "my_table",
    metadata,
    Column("id", String, primary_key=True),
    Column("other_column", String)
)
source = sql_table(
    my_sqlalchemy_engine,
    metadata=metadata,
    table="my_table",
    included_columns=["id", "other_column"]
)
@tommy1x tommy1x changed the title Query MSSQL source with no system columns Query MSSQL source with no system tables Feb 3, 2025
@tommy1x tommy1x changed the title Query MSSQL source with no system tables Load MSSQL source with no system tables Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

1 participant