[BUG] arm image can't connect to postgres using scram-sha-256 for password encryption

[maxwell-gobel]

Agent Environment
Image Tag: gcr.io/datadoghq/agent:7.41.1
Image Digest: gcr.io/datadoghq/agent@sha256:4992b38387ca6dbb732f3493ed66cdf5efe7e08154b7e9e4159b1735dbaabe3f
Image Architecture:

❯ docker inspect gcr.io/datadoghq/agent@sha256:4992b38387ca6dbb732f3493ed66cdf5efe7e08154b7e9e4159b1735dbaabe3f | jq '.[0].Architecture'
"arm64"

❯ agent version
Agent 7.41.1 - Commit: 4f39b9e - Serialization version: v5.0.39 - Go version: go1.18.9

Describe what happened:
Migrated our k8s nodes from amd64 to arm64, and a previously functional postgres custom query check started failing with:

Error: SCRAM authentication requires libpq version 10 or above
      Traceback (most recent call last):
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/base/checks/base.py", line 1122, in run
          self.check(instance)
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/postgres/postgres.py", line 608, in check
          raise e
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/postgres/postgres.py", line 582, in check
          self._connect()
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/postgres/postgres.py", line 411, in _connect
          self.db = self._new_connection(self._config.dbname)
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/postgres/postgres.py", line 395, in _new_connection
          conn = psycopg2.connect(**args)
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
          conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
      psycopg2.OperationalError: SCRAM authentication requires libpq version 10 or above

❯ docker run -it --platform=linux/amd64 --entrypoint python gcr.io/datadoghq/agent:7.41.1 -c "import psycopg2; print(psycopg2.__libpq_version__)"
110005

❯ docker run -it --platform=linux/arm64 --entrypoint python gcr.io/datadoghq/agent:7.41.1 -c "import psycopg2; print(psycopg2.__libpq_version__)"
90425

Describe what you expected:

• postgres check to support authenticating using the default password encryption mechanism to the minimal supported postgres version (11) for both x86 and arm architectures

Steps to reproduce the issue:
Using the linux/arm64 gcr.io/datadoghq/agent:7.41.1 image, configure a postgres custom query check using password auth
Also reproduces with 7.42.0

Additional environment details (Operating System, Cloud provider, etc):
Running using the datadog-2.37.4 helm chart. Custom query check is configured on the cluster-agent and farmed out to other agents (not running a dedicated cluster checks runner)

Related:

• Upstream psycopg2-binary issue
  • Comment on related psycopg2 issue
• Same issue on M1s
  • Can't use the same workaround implicilty suggested here for M1s, since our EKS nodes don't have an emulation layer available

[AmitArie]

any updates on this issue? 🙏

[DmytroKozlovskyi]

any updates on this issue? 🙏

This is actually fixed as I know in version 7.47.0. Our organization already has updated and doesn't see this error anymore.

[jmeagher]

I just ran into the same error. I updated from Agent 7.43.2 - Commit: 26255a9 - Serialization version: v5.0.67 - Go version: go1.19.6 to Agent 7.49.0 - Commit: 9cb8e9c - Serialization version: v5.0.97 - Go version: go1.20.10 and it's working for me.

[FlorentClarret]

Hello 👋 Closing this issue as it has been fixed in newer releases. Feel free to open it again if needed 🙂