Cannot tunnel TLS connection through TLS connection

[MaximilianBlochberger]

BPO	29394
Nosy	@terryjreedy, @tiran, @carlbordum
PRs	gh-73580: Docs for tunnelling TLS through TLS #22539

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-01-31.07:19:43.632>
labels = ['type-bug', '3.7', 'docs']
title = 'Cannot tunnel TLS connection through TLS connection'
updated_at = <Date 2020-10-04.13:23:03.746>
user = 'https://bugs.python.org/MaximilianBlochberger'

bugs.python.org fields:

activity = <Date 2020-10-04.13:23:03.746>
actor = 'carlbordum'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2017-01-31.07:19:43.632>
creator = 'Maximilian Blochberger'
dependencies = []
files = []
hgrepos = []
issue_num = 29394
keywords = ['patch']
message_count = 6.0
messages = ['286517', '286519', '286787', '286897', '286928', '301507']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'christian.heimes', 'docs@python', 'Maximilian Blochberger', 'carlbordum']
pr_nums = ['22539']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29394'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

Linked PRs

• [3.12] gh-73580: Docs for tunnelling TLS through TLS (GH-22539) #116149
• [3.11] gh-73580: Docs for tunnelling TLS through TLS (GH-22539) #116150

[MaximilianBlochberger]

I have the following scenario: Client → Proxy → Target.

The following two scenarios are working perfectly fine:

1. Establishing a TLS-secured connection to the proxy and then tunnel traffic through that connection to the target. This results in the proxy being able to observe and manipulate the traffic in both directions. It protects against an adversary who has no control over the proxy, e.g. it prevents observers from learning that you are using a proxy (if the IP/port is not known) and from reading the actual traffic.

2. Establish a non-secured connection to the proxy and then tunnel TLS-secured traffic through that connection to the target. That prevents the proxy from being able to observe or manipulate the traffic. Although an observer could learn that you are using a proxy and what target you are connecting to.

Now what I tried was to establish a TLS-secured connection to the proxy and then to establish a TLS-secured tunnel to the target, effectively resulting in two layers of TLS in between the client and the proxy. This would protect from an observer learning that you are using a proxy and where you connect to (the proxy still knows) but preventing the proxy from observing and manipulating the actual traffic to the target.

This does not work in Python 3.6. The TLS-secured connection to the proxy is straight forward and can be easily done with ssl.SSLContext.wrap_socket(). The TCP connection between the proxy and the target can then be established by issuing an HTTP CONNECT request. The response can than be read without closing the connection as done in http.client.HTTPConnection._tunnel(). Now my idea was to call ssl.SSLContext.wrap_socket() again (with a different context for the target) and send traffic through that. Unfortunately the TLS handshake fails with the error message "unknown protocol". I looked into the actual traffic transmitted and realised that the handshake was performed in plain text – so effectively stripping the TLS layer that was established already – which results in the proxy server not knowing how to handle the traffic (as it is not TLS-secured) aborting the connection (and reporting a fatal TLS alert).

This leads to the conclusion that another call to ssl.SSLContext.wrap_socket() will override a previous call of the same function (different context object though). I think this is unexpected behaviour.

It might be easier to handle such scenarios if a tunnel would be a separate http.client.HTTP(S)Connection object, see issue bpo-24964. This would also allow to handle ssl-specific calls such as ssl.SSLSocket.getpeercert() as each layer probably uses different certificates.

[tiran]

You cannot use wrap_socket() to wrap a SSLSocket into another SSLSocket. Python uses SSL_set_fd() to wrap the socket's file descriptor. OpenSSL directly pulls and pushes data to the connection. Instead you have to use wrap_bio() and do I/O on your own.

[MaximilianBlochberger]

Okay, I see, thanks for the hint. That worked perfectly – I found asyncio.sslproto._SSLPipe very useful for that purpose.

I personally consider the behaviour of ssl.SSLContext.wrap_socket() unexpected and would raise an exception if that method call is tried on an instance of ssl.SSLSocket. But as this would be a change that could lead to backwards compatibility issues (if developers depend on that behaviour) this is probably not a good idea.

I think that the documentation for ssl.SSLContext.wrap_socket() has this behaviour to avoid future confusion – including a hint to make use of ssl.SSLContext.wrap_bio() instead.

[terryjreedy]

Maximilian, I could not tell if you are still requesting that something be changed, or if this should be closed.

[MaximilianBlochberger]

Yes. There should be at least an explanation of this behaviour in the documentation of the wrap_socket() function.

I would additionally raise an exception if wrap_socket() is called and a socket is passed that is already wrapped. But I'm not sure if that is considered as an acceptable choice, as I am unfamiliar with Python development.

[tiran]

A documentation update wouldn't hurt, though.

[kevinAlbs]

This may be a duplicate of #89168.