Long column names considered repeated because their names got truncated in code

[isabela-angelo-ze-delivery]

Affected module
Ingestion

Describe the bug
I have a table in Glue Metabase with two columns:
installments_installment_chargeback_refunds_chargeback_refund_payment_id
installments_installment_chargeback_refunds_chargeback_refund_payment_date

And during the ingestion I get the following error:
Failed to ingest CreateTableRequest [***] due to api request failure: Column name installments_installment_chargeback_refunds_chargeback_refund_pa is repeated

To Reproduce

Create a table in Glue Metadata with a name longer than 64 characteres and with the same characteres at the begining 64 chars.
Run the ingestion process in Airfllow.

Expected behavior
Columns aren't repeated so the error shouldn't happen.

Version:

• OS: debian:bullseye-slim ( image python:3.9.7-slim-bullseye )
• Python version: 3.9.7
• OpenMetadata version: 0.13.2
• OpenMetadata Ingestion package version: openmetadata-ingestion[glue]==0.13.2.8

Additional context
I have been checking the code and found the line where the column name is considered as just 64 chars:
https://github.com/open-metadata/OpenMetadata/blob/26db825b7195000c9a550cbec889f985a024ac97/ingestion/src/metadata/ingestion/source/database/glue/metadata.py#LL298C46-L298C48

I know it's not nice to keep increasing this number so maybe there is a way to deduplicate these column names in Column class.
What do you think?

[SuperBo]

this one also affects databricks ingestion with unity catalog

OpenMetadata/ingestion/src/metadata/ingestion/source/database/databricks/unity_catalog/metadata.py

Line 389 in d263d67

parsed_string["name"] = column.name[:64]

[harshach]

@SuperBo @isabela-angelo-ze-delivery this is now fixed. Please giving it a try again. cc @pmbrull

[SuperBo]

Hi @harshach, @pmbrull this issue happens again with recent version "1.1.5" and "1.1.6" for databricks.

The latest source code (main) reflects this.

https://github.com/open-metadata/OpenMetadata/blob/5a3d759b48d93647fefd7cc9bc1399a28b44e67f/ingestion/src/metadata/ingestion/source/database/databricks/unity_catalog/metadata.py#L463C16-L463C16