No way to tolerate SSLError complaining about self-signed certificate

[andy-maier]

Describe the bug

Creating a JIRA object for a Jira server that is set up with a self-signed certificate causes requests to raise:

SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))]

I did not find a way to configure the jira client to tolerate that.

To Reproduce
Steps to reproduce the behavior:

1. Have a Jira server that is set up with a self-signed certificate.
2. Execute the following Python code:

my_jira = jira.JIRA(server=MY_JIRA_BASE_URL, basic_auth=(MY_JIRA_EMAIL, MY_JIRA_TOKEN))

3. This fails with the stack trace shown below.

Expected behavior

There is a way to have the jira client tolerate the SSLError for the self-signed certificate.

Stack Trace
The actual server DNS hostname has been replaced with <jira_host>.

WARNING:root:HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'))) while doing GET https://<jira_host>:8443/rest/api/2/serverInfo
WARNING:root:Got ConnectionError [HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))] errno:None on GET https://<jira_host>:8443/rest/api/2/serverInfo
{'response': None, 'request': <PreparedRequest [GET]>}
{'response': None, 'request': <PreparedRequest [GET]>}
WARNING:root:Got recoverable error from GET https://<jira_host>:8443/rest/api/2/serverInfo, will retry [1/3] in 9.7132500124177s. Err: HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
Debug: requests: Auth='Basic bWFpZXJhQGRlLmlibS5jb206bW8tYXBpLXpRc0JQTUJEdjd3RFJLWVlvWXFSVXN3OQ=='
WARNING:root:HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'))) while doing GET https://<jira_host>:8443/rest/api/2/serverInfo
WARNING:root:Got ConnectionError [HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))] errno:None on GET https://<jira_host>:8443/rest/api/2/serverInfo
{'response': None, 'request': <PreparedRequest [GET]>}
{'response': None, 'request': <PreparedRequest [GET]>}
WARNING:root:Got recoverable error from GET https://<jira_host>:8443/rest/api/2/serverInfo, will retry [2/3] in 15.12495398342077s. Err: HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
Debug: requests: Auth='Basic bWFpZXJhQGRlLmlibS5jb206bW8tYXBpLXpRc0JQTUJEdjd3RFJLWVlvWXFSVXN3OQ=='
WARNING:root:HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'))) while doing GET https://<jira_host>:8443/rest/api/2/serverInfo
WARNING:root:Got ConnectionError [HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))] errno:None on GET https://<jira_host>:8443/rest/api/2/serverInfo
{'response': None, 'request': <PreparedRequest [GET]>}
{'response': None, 'request': <PreparedRequest [GET]>}
WARNING:root:Got recoverable error from GET https://<jira_host>:8443/rest/api/2/serverInfo, will retry [3/3] in 24.89636262732621s. Err: HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
Debug: requests: Auth='Basic bWFpZXJhQGRlLmlibS5jb206bW8tYXBpLXpRc0JQTUJEdjd3RFJLWVlvWXFSVXN3OQ=='
WARNING:root:HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'))) while doing GET https://<jira_host>:8443/rest/api/2/serverInfo
Traceback (most recent call last):
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/connection.py", line 411, in connect
    self.sock = ssl_wrap_socket(
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/Cellar/[email protected]/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/Cellar/[email protected]/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/usr/local/Cellar/[email protected]/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maiera/virtualenvs/iaas39/bin/check_aha_items", line 33, in <module>
    sys.exit(load_entry_point('check-aha-items', 'console_scripts', 'check_aha_items')())
  File "/Users/maiera/Projects/ZaaS/repos/IaaS/tools/check_aha_items/check_aha_items/check_aha_items.py", line 656, in main
    features, date_dt = get_aha_features(options)
  File "/Users/maiera/Projects/ZaaS/repos/IaaS/tools/check_aha_items/check_aha_items/check_aha_items.py", line 543, in get_aha_features
    aha_features = retrieve_aha_features(options)
  File "/Users/maiera/Projects/ZaaS/repos/IaaS/tools/check_aha_items/check_aha_items/check_aha_items.py", line 406, in retrieve_aha_features
    vpc_jira = jira.JIRA(server=VPC_JIRA_BASE_URL,
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/jira/client.py", line 491, in __init__
    si = self.server_info()
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/jira/client.py", line 2789, in server_info
    j = self._get_json("serverInfo")
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/jira/client.py", line 3422, in _get_json
    r = self._session.get(url, params=params)
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/jira/resilientsession.py", line 196, in get
    return self.__verb("GET", str(url), **kwargs)
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/jira/resilientsession.py", line 189, in __verb
    raise exception
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/jira/resilientsession.py", line 166, in __verb
    response = method(url, timeout=self.timeout, **kwargs)
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/Users/maiera/virtualenvs/iaas39/lib/python3.9/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='<jira_host>', port=8443): Max retries exceeded with url: /rest/api/2/serverInfo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))

Version Information
Type of Jira instance:

• Jira Cloud (Hosted by Atlassian)
• Jira Server or Data Center (Self-hosted)
  Python Interpreter: Python 3.9.7
  jira-python: master
  OS: macOS
  IPython (Optional): N/A
  Other Dependencies: requests 2.26.0, urllib3 1.26.6

Additional context
N/A

[xulfus]

We have experienced similar issues. As an example

from jira import JIRA
import re
JIRA_API_GW_TOKEN = 'xxxxxxxxx'
headers = JIRA.DEFAULT_OPTIONS["headers"].copy()
headers["X-APIGWAuth"] = f"Bearer {JIRA_API_GW_TOKEN}"
options= {"headers": headers, "server": "https://jiraserver:443/jira", 'verify': './chain.pem'}
jira=JIRA(options, basic_auth=("xxx", "xxx"), get_server_info=False)

# Create ticket succeeds
fields = {
	"customfield_123": {"value": "Some Value"}
	# fields redacted
}
ticket = jira.create_issue(fields=fields)
print("Ticket created  \n {}".format(ticket))

# Update ticket fails
issue = jira.issue('PR-1234')
issue.update(fields={'customfield_123': {'value': 'New Value'}})

The mysterious thing is that create_issue here succeeds, while issue.update fails with
WARNING:root:HTTPSConnectionPool(host='<redacted>', port=443): Max retries exceeded with url: /rest/api/2/issue/1293819 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'))) while doing PUT http://<redacted>

When disabling verification with options= {"headers": headers, "server": "https://jiraserver:443/jira", 'verify': False} the update fails silently.

BUT the update can be performed using requests directly, with both verify enabled and disabled. This is succesful:

from requests.auth import HTTPBasicAuth
import requests
import json
JIRA_API_GW_TOKEN = 'xxxxxxxxx'
headers = JIRA.DEFAULT_OPTIONS["headers"].copy()
headers["X-APIGWAuth"] = f"Bearer {JIRA_API_GW_TOKEN}"
data={"fields": {'customfield_123': {'value': 'New Value'}}}   
url = 'https://jiraserver:443/jira/rest/api/2/issue/PR-1234'
payload = json.dumps(data)
resp = requests.put(url, headers=headers, auth=HTTPBasicAuth('xxxx', 'xxxx'), data=payload, verify='./chain.pem')`

NB we are interfacing an API gateway and have no access to logs for either the gateway or the target Jira server.

[kchason]

I am in the process of testing a fix for this that allows you to provide the certificate authority for the server certificate and hope to have a pull request in tonight.

If you want to just skip verification, you can do that by setting options['verify'] to False in the JIRA object constructor.

https://github.com/pycontribs/jira/blob/main/jira/client.py#L311

[adehad]

from #1202, we now allow align with requests with respect to the verify argument.