SSL/TLS server certificate validation #1202
Comments
patatetom
added
bug
|
@patatetom can you provide the example website for us to reproduce this problem? |
isidentical
changed the title
isidentical
added
awaiting-response
and removed
new
|
hi @isidentical, |
|
hi, first of all, I didn't manage to simulate the problem :
so i rechecked my configuration :
openssl verify -CAfile my.root.cert.pem -untrusted my.intermediate.cert.pem my.server.cert.pem
my.server.cert.pem: OK
trust list | egrep -i 'label:…'
server=…
curl --disable --head https://$server/ && echo OK
HTTP/2 200
server: nginx
date: Wed, 19 Jan 2022 10:14:49 GMT
content-type: text/html; charset=UTF-8
content-length: 5076
last-modified: Tue, 04 Feb 2020 15:39:52 GMT
etag: "5e399048-13d4"
strict-transport-security: max-age=31536000 ;includeSubDomains; preload
x-frame-options: deny
accept-ranges: bytes
OK
wget --server-response --output-document=/dev/null --quiet https://$server/ && echo OK
SSL_INIT
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 19 Jan 2022 10:19:57 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 5076
Last-Modified: Tue, 04 Feb 2020 15:39:52 GMT
Connection: keep-alive
ETag: "5e399048-13d4"
Strict-Transport-Security: max-age=31536000 ;includeSubDomains; preload
X-Frame-Options: deny
Accept-Ranges: bytes
OK
python -m venv test.httpie
source test.httpie/bin/activate
pip list
Package Version
---------- -------
pip 21.2.4
setuptools 58.1.0
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/tmp/ssl/test.httpie/bin/python -m pip install --upgrade pip' command.
pip install --upgrade pip
Requirement already satisfied: pip in ./test.httpie/lib/python3.10/site-packages (21.2.4)
Collecting pip
Using cached pip-21.3.1-py3-none-any.whl (1.7 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 21.2.4
Uninstalling pip-21.2.4:
Successfully uninstalled pip-21.2.4
Successfully installed pip-21.3.1
pip install httpie
Collecting httpie
Using cached httpie-2.6.0-py3-none-any.whl (63 kB)
Requirement already satisfied: setuptools in ./test.httpie/lib/python3.10/site-packages (from httpie) (58.1.0)
Collecting requests-toolbelt>=0.9.1
Using cached requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
Collecting defusedxml>=0.6.0
Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting Pygments>=2.5.2
Using cached Pygments-2.11.2-py3-none-any.whl (1.1 MB)
Collecting requests[socks]>=2.22.0
Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting charset-normalizer>=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting PySocks!=1.5.7,>=1.5.6
Using cached PySocks-1.7.1-py3-none-any.whl (16 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests, PySocks, requests-toolbelt, Pygments, defusedxml, httpie
Successfully installed PySocks-1.7.1 Pygments-2.11.2 certifi-2021.10.8 charset-normalizer-2.0.10 defusedxml-0.7.1 httpie-2.6.0 idna-3.3 requests-2.27.1 requests-toolbelt-0.9.1 urllib3-1.26.8
pip list
Package Version
------------------ ---------
certifi 2021.10.8
charset-normalizer 2.0.10
defusedxml 0.7.1
httpie 2.6.0
idna 3.3
pip 21.3.1
Pygments 2.11.2
PySocks 1.7.1
requests 2.27.1
requests-toolbelt 0.9.1
setuptools 58.1.0
urllib3 1.26.8
https --header https://$server/ && echo OK
https: error: SSLError: HTTPSConnectionPool(host='…', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) while doing a GET request to URL: https://…/
python <<~~~~
import http.client
server="$server"
conn = http.client.HTTPSConnection(server)
conn.request('GET', '/')
resp = conn.getresponse()
print(resp.status)
print(resp.headers.items())
~~~~
200
[('Server', 'nginx'), ('Date', 'Wed, 19 Jan 2022 10:27:52 GMT'), ('Content-Type', 'text/html; charset=UTF-8'), ('Content-Length', '5076'), ('Last-Modified', 'Tue, 04 Feb 2020 15:39:52 GMT'), ('Connection', 'keep-alive'), ('ETag', '"5e399048-13d4"'), ('Strict-Transport-Security', 'max-age=31536000 ;includeSubDomains; preload'), ('X-Frame-Options', 'deny'), ('Accept-Ranges', 'bytes')]
python <<~~~~
import requests
req = requests.get('https://$server/')
print(req.status_code)
~~~~
Traceback (most recent call last):
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._validate_conn(conn)
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
conn.connect()
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/connection.py", line 416, in connect
self.sock = ssl_wrap_socket(
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.10/ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib/python3.10/ssl.py", line 1070, in _create
self.do_handshake()
File "/usr/lib/python3.10/ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/requests/adapters.py", line 440, in send
resp = conn.urlopen(
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
retries = retries.increment(
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='…', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/requests/sessions.py", line 529, in request
resp = self.send(prep, **send_kwargs)
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/requests/sessions.py", line 645, in send
r = adapter.send(request, **kwargs)
File "/tmp/ssl/test.httpie/lib/python3.10/site-packages/requests/adapters.py", line 517, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='…', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)')))I hope all this can help... |
|
some additional tests for https --verify=my.root.cert.pem --header https://$server/ && echo OK
HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: keep-alive
Content-Length: 5076
Content-Type: text/html; charset=UTF-8
Date: Wed, 19 Jan 2022 11:23:28 GMT
ETag: "5e399048-13d4"
Last-Modified: Tue, 04 Feb 2020 15:39:52 GMT
Server: nginx
Strict-Transport-Security: max-age=31536000 ;includeSubDomains; preload
X-Frame-Options: deny
OK
https --ssl=ssl2.3 --header https://$server/ && echo OK
https: error: SSLError: HTTPSConnectionPool(host='…', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) while doing a GET request to URL: https://…/
https --ssl=tls1 --header https://$server/ && echo OK
https: error: SSLError: HTTPSConnectionPool(host='…', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) while doing a GET request to URL: https://…/
https --ssl=tls1.1 --header https://$server/ && echo OK
https: error: SSLError: HTTPSConnectionPool(host='…', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) while doing a GET request to URL: https://…/
https --ssl=tls1.2 --header https://$server/ && echo OK
https: error: SSLError: HTTPSConnectionPool(host='…', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) while doing a GET request to URL: https://…/ |
|
Fantastic analysis of the problem. I was bitten by it just yesterday, as well. Interestingly, HTTPie has been working for several months on the identical infrastructure. The SSL certificate, managed by Let's Encrypt, doesn't seem to have been updated and the httpie version is the same since the project's inception (aka creation of the GitLab CI pipeline) earlier this year.
What did change yesterday, when the issue started to occur, was the I got this fixed by telling export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt |
|
The same SSL issue also recently came up after release new alpine version in |
|
httpie has been broken on all of my arch linux machines for months now due to this issue.
|
This is essentially an issue of the It looks like the issue has been addressed in the last two releases (3.2.3 and 3.2.4) via 7f03c52 and fd30c4e, but at least the Have you tried adding an export of this variable to your |
|
I have this problem with latest httpie $ curl https://posh-condor-88.deno.dev
Hello, vccv!but $ http https://posh-condor-88.deno.dev
http: error: SSLError: HTTPSConnectionPool(host='posh-condor-88.deno.dev', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)'))) while doing a GET request to URL: https://posh-condor-88.deno.dev/so $ http --verify=no https://posh-condor-88.deno.dev
HTTP/1.1 200 OK
content-length: 12
content-type: text/plain;charset=UTF-8
date: Tue, 12 Nov 2024 09:13:35 GMT
server: deno/gcp-europe-west3
vary: Accept-Encoding
via: http/1.1 edgeproxy-h
Hello, vccv!I was fixed by but maybe we can do it by default during installation, or at leas add HUGE WARNING to installation script that this should be added, or maybe by default disable ssl verification and dispaly yellow warning for any httpie call that this line have to added to ~/.bashrc / ~/.zshrc Yesterday I lost too much time by this error, i belive that there is workaround on |
Checklist
Context
Minimal reproduction code and steps
https https://serverhttps --verify rootCA.pem https://serverCurrent result
https: error: SSLError: HTTPSConnectionPool(host='server', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'))) while doing a GET request to URL: https://server/Expected result
Debug output
Additional information, screenshots, or code examples
hi,
it seems that it is the use of the custom SSL context which poses problem because the direct use of the python ssl module does not pose any particular problem to access
https://server/...regards, lacsaP.
The text was updated successfully, but these errors were encountered: