Option to ignore SSL certificate check for Elasticsearch connector

[alogvinov-forter]

Hi team,

Thank you very much for adding the support for Elasticsearch in the latest release! We've been waiting for this connector for quite some time now, and after upgrade to 1.2 we should finally able to ingest this data source into OpenMetadata.
While setting up the ingestion and testing the connection, I ran into an issue with self-signed SSL certificates:

TLS error caused by: TlsError(TLS error caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] 
certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)))

I understand there is the "Client Certificate Path" field which can be used to supply the certificate, but I'm wondering if there is a way to skip verification of certificates altogether? I tried passing verify_certs=False in the advanced config, it didn't work. The problem is, we have ~30 Elasticsearch clusters all using self-signed certificates which are also rotated periodically. So maintaining all of this configuration is going to be a nightmare.

I found this GitHub issue - elastic/elasticsearch-py#712 - and tried following the suggested options there with no luck. Any advice would be very much appreciated, thanks! 🙏

[alogvinov-forter]

We have an internal Python client for Elasticsearch which basically wraps up the default one, and this is the piece that seems to be relevant:

ssl_context = create_ssl_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
connection_kwargs.update({"ca_certs": False, "ssl_context": ssl_context})