Skip to content

Commit

Permalink
chore: Update docstring for DatabaseInfo in OpenLineage provider
Browse files Browse the repository at this point in the history
Signed-off-by: Kacper Muda <[email protected]>
  • Loading branch information
kacpermuda committed Jan 14, 2025
1 parent 3d7a2f7 commit e1baddd
Showing 1 changed file with 16 additions and 14 deletions.
30 changes: 16 additions & 14 deletions providers/src/airflow/providers/openlineage/sqlparser.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,24 +110,26 @@ class DatabaseInfo:
metadata from more than one database. For instance, in Snowflake or MS SQL
(where each database is treated as a top-level namespace), you might have a query like:
SELECT ...
FROM db1.information_schema.columns
...
UNION ALL
SELECT ...
FROM db2.information_schema.columns
...
```
SELECT ...
FROM db1.information_schema.columns WHERE ...
UNION ALL
SELECT ...
FROM db2.information_schema.columns WHERE ...
```
In Redshift, setting this to True together with `use_flat_cross_db_query=True` allows
adding database filters to the query, for example:
SELECT ...
FROM SVV_REDSHIFT_COLUMNS
WHERE
SVV_REDSHIFT_COLUMNS.database == db1 # This is skipped when False
AND SVV_REDSHIFT_COLUMNS.schema == schema1
AND SVV_REDSHIFT_COLUMNS.table IN (table1, table2)
OR ...
```
SELECT ...
FROM SVV_REDSHIFT_COLUMNS
WHERE
SVV_REDSHIFT_COLUMNS.database == db1 # This is skipped when False
AND SVV_REDSHIFT_COLUMNS.schema == schema1
AND SVV_REDSHIFT_COLUMNS.table IN (table1, table2)
OR ...
```
However, certain databases (e.g., PostgreSQL) do not permit true cross-database queries.
In such dialects, enabling cross-database support may lead to errors or be unnecessary.
Expand Down

0 comments on commit e1baddd

Please sign in to comment.