Teradata Connector

[gpby]

Supporting Teradata Vantage:

Fixes #15756

Here is a beta version of Teradata Database / Teradata Vantage Connector
It supports only Metadata and Profiler ingestion

Known limitations:

• No columns info for Views (only ColumnName)
• Unable to parse DDL for views for some cases: REPLACE VIEW (only CREATE VIEW supported), LOCKING .. FOR ACEESS, etc. Third party library sqlparse should be adopted for Teradata specific.
• There are no schemas in Teradata only Databases but from entity point of view Teradara Database is OpenMetadata Schema. So Database for each Teradata object in OM will be default and Terdata DatabaseName used as OM Schema

Type of change:

• New feature

Checklist:

• I have read the CONTRIBUTING document.

• My PR title is Fixes 15756: Support Teradata Connector

• I have commented on my code, particularly in hard-to-understand areas.

• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

• The issue properly describes why the new feature is needed, what's the goal, and how we are building it. Any discussion
  or decision-making process is reflected in the issue.

• I have updated the documentation.

• I have added tests around the new logic.

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

Jest test Coverage

UI tests summary

Lines	Statements	Branches	Functions
	64.79% (34797/53710)	41.66% (13718/32927)	43.6% (4268/9788)

[cypress]

2 failed tests on run #34268 ↗︎

2

481

8

0

0

Details:

Merge branch 'main' into teradata
Project: openmetadata	Commit: 966fe39fae
Status: Failed	Duration: 00:14 💡
Started: May 27, 2024 7:40 AM	Ended: May 27, 2024 8:40 AM

  cypress/e2e/Pages/Services.spec.ts • 2 failed tests • cypress-ci-mysql-pr-9250839733-1

View Output Video

Test		Artifacts
Services detail page > Database Service page > GlossaryTerm assign, update & delete		Screenshots Video
Services detail page > Database Service page > Soft delete		Screenshots Video

Review all test suite changes for PR #16373 ↗︎

[pmbrull]

hi @gpby, thanks a lot for the contribution! Appreciate the effort!

A couple of comments here:

• You might need to run make py_format to ensure CI passes
• We might need your help with the docs. For example, here you can find the documentation for Athena. Basically, we need to ensure we direct users to the proper permissions and field descriptions. This can be a followup PR.
• Re the lineage limitations, if you could please share a few queries as a comment in this issue Lineage Query Parsing issues #7427, it'll help us make the necessary enhancements to the lineage parsing module to add proper support for Teradata's dialect

Thanks!

[gpby]

1. I've done make py_format
2. I'm still working on documentattion
3. I'll add sql examples with parsing errors

[pmbrull]

Thanks @gpby, this LGTM. We can add the docs to another PR. Appreciate the effort

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ui'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[sonarqubecloud]

Quality Gate failed for 'open-metadata-ingestion'

Failed conditions
6.6% Coverage on New Code (required ≥ 20%)

See analysis details on SonarCloud