urllib3 compatibility: signature change of HTTPConnectionPool._new_conn breaks other subclasses

[lilatomic]

urllib3 compatibility: signature change of HTTPConnectionPool._new_conn breaks other subclasses

urllib3.connectionpool.HTTPConnectionPool method _new_conn had it's signature changed from def _new_conn(self) -> HTTPConnection to def _new_conn(self, *, heb_timeout: Timeout | None = None) -> HTTPConnection. Importantly, this adds the kwarg-only heb_timeout.
This causes a problem with subclasses of HTTPConnectionPool, such as the "docker" package's UnixHTTPConnectionPool (source code for UnixHTTPConnectionPool). Since this kwarg doesn't exist on that function, we get "TypeError: _new_conn() got an unexpected keyword argument 'heb_timeout'".

I think the easiest thing to do is to try to get the "docker" or upstream "urllib3" package to modify the signature to def _new_conn(self, **kwargs) -> HTTPConnection.

Environment

I've tested with 'docker==7.1.0' and 'docker==7.0.0', but I believe it affects most of them

>>> import platform
>>> import ssl
>>> import urllib3
>>> 
>>> print("OS", platform.platform())
OS Linux-6.5.0-35-generic-x86_64-with-glibc2.35
>>> print("Python", platform.python_version())
Python 3.10.12
>>> print(ssl.OPENSSL_VERSION)
OpenSSL 3.0.2 15 Mar 2022
>>> print("urllib3", urllib3.__version__)
urllib3 2.7.912

Steps to Reproduce

python3 -m venv /tmp/venv && source /tmp/venv/bin/activate
# note that urllib3 will be pulled in as a dependency of "docker"
pip3 install 'docker==7.1.0' 'urllib3==2.2.1' 'urllib3-future==2.7.912'

# force docker to use unix socket
export DOCKER_HOST="unix:///run/user/1000/docker.sock"
python3 -c 'import docker; docker.from_env()'

Expected Behavior

"urllib3-future" should be a drop-in replacement for "urllib3"

Actual Behavior

traceback

``` python3 -c 'import docker; docker.from_env()' Traceback (most recent call last): File "/tmp/venv/lib/python3.10/site-packages/docker/api/client.py", line 223, in _retrieve_server_version return self.version(api_version=False)["ApiVersion"] File "/tmp/venv/lib/python3.10/site-packages/docker/api/daemon.py", line 181, in version return self._result(self._get(url), json=True) File "/tmp/venv/lib/python3.10/site-packages/docker/utils/decorators.py", line 44, in inner return f(self, *args, **kwargs) File "/tmp/venv/lib/python3.10/site-packages/docker/api/client.py", line 246, in _get return self.get(url, **self._set_request_timeout(kwargs)) File "/tmp/venv/lib/python3.10/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) File "/tmp/venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/tmp/venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/tmp/venv/lib/python3.10/site-packages/requests/adapters.py", line 589, in send resp = conn.urlopen( File "/tmp/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1366, in urlopen conn = self._get_conn(timeout=pool_timeout, heb_timeout=timeout_obj) File "/tmp/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 530, in _get_conn return conn or self._new_conn(heb_timeout=heb_timeout) TypeError: UnixHTTPConnectionPool._new_conn() got an unexpected keyword argument 'heb_timeout'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 1, in
File "/tmp/venv/lib/python3.10/site-packages/docker/client.py", line 94, in from_env
return cls(
File "/tmp/venv/lib/python3.10/site-packages/docker/client.py", line 45, in init
self.api = APIClient(*args, **kwargs)
File "/tmp/venv/lib/python3.10/site-packages/docker/api/client.py", line 207, in init
self._version = self._retrieve_server_version()
File "/tmp/venv/lib/python3.10/site-packages/docker/api/client.py", line 230, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: UnixHTTPConnectionPool._new_conn() got an unexpected keyword argument 'heb_timeout'

</details>

[Ousret]

Indeed, we have something incompatible. They override a private method, thus the unexpected error.
We took action to fix this issue. based on the reproducer, the issue is fixed. We went further on by testing other features.

I think the easiest thing to do is to try to get the "docker" or upstream "urllib3" package to modify the signature to..

urllib3.future is meant to be transparent, and most of the time, "they" have nothing to do, we have.
run pip install urllib3-future -U to upgrade to latest.

regards,

[lilatomic]

thanks for the super-fast turnaround! Works great!