You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If we have misalignment in the ingestion pipeline JSON file that gets generated for the ingestion from OM UI and the pydantic models, the errors are hard to understand:
IngestionPipeline expected: 170 validation errors for IngestionPipeline
sourceConfig -> config -> viewParsingTimeoutLimit
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> type
value is not a valid enumeration member; permitted: 'DatabaseUsage' (type=type_error.enum; enum_values=[<DatabaseUsageConfigType.DatabaseUsage: 'DatabaseUsage'>])
sourceConfig -> config -> databaseFilterPattern
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> includeStoredProcedures
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> includeTables
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> includeTags
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> includeViews
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> markDeletedTables
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> queryParsingTimeoutLimit
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> schemaFilterPattern
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> tableFilterPattern
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> useFqnForFiltering
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> viewParsingTimeoutLimit
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> type
value is not a valid enumeration member; permitted: 'DatabaseLineage' (type=type_error.enum; enum_values=[<DatabaseLineageConfigType.DatabaseLineage: 'DatabaseLineage'>])
sourceConfig -> config -> includeStoredProcedures
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> includeTables
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> includeTags
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> includeViews
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> markDeletedTables
extra fields not permitted (type=value_error.extra)
sourceConfig -> config -> queryParsingTimeoutLimit
extra fields not permitted (type=value_error.extra)
The goal of this ticket is to use the same parse_gracefully logic we have for workflows but at the Ingestion Pipeline level on the Airflow APIs.
The text was updated successfully, but these errors were encountered:
If we have misalignment in the ingestion pipeline JSON file that gets generated for the ingestion from OM UI and the pydantic models, the errors are hard to understand:
The goal of this ticket is to use the same
parse_gracefully
logic we have for workflows but at the Ingestion Pipeline level on the Airflow APIs.The text was updated successfully, but these errors were encountered: