Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mixed protocol (IPv6 -> IPv4) PROXY headers in multiplexer #39205

Open
webvictim opened this issue Mar 11, 2024 · 2 comments
Open

Support mixed protocol (IPv6 -> IPv4) PROXY headers in multiplexer #39205

webvictim opened this issue Mar 11, 2024 · 2 comments
Assignees
Labels
c-dd Internal Customer Reference feature-request Used for new features in Teleport, improvements to current should be #enhancements ipv6 networking Network connectivity features/problems tls-routing Issues related to TLS routing

Comments

@webvictim
Copy link
Contributor

What would you like Teleport to do?

Support an IPv6 source header connecting to an IPv4 target as part of PROXY protocol headers.

Currently, when traffic ingresses to Teleport via an IPv6 source address (for example an IPv6-capable host connecting to an proxied IPv6 edge POP on Cloudflare) but the Teleport server only accepts connections on an IPv4 address (such the Teleport origin being on a home connection without IPv6 support), Teleport rejects the connection:

Mar 08 15:42:36 artemis teleport[1817446]: 2024-03-08T15:42:36-04:00 DEBU [SSH:PROXY] Incoming connection [2600:3c06::f03d:91ff:fe73:1fc6]:23252 -> 192.168.64.254:443 version: SSH-2.0-Go, certtype: "host" pid:1817446.1 sshutils/server.go:538
Mar 08 15:42:36 artemis teleport[1817446]: 2024-03-08T15:42:36-04:00 DEBU [PROXY:1]   Received transport request. pid:1817446.1 reversetunnel/srv.go:673
Mar 08 15:42:36 artemis teleport[1817446]: 2024-03-08T15:42:36-04:00 ERRO [PROXY:1]   Failed to create signed PROXY header. error:[
Mar 08 15:42:36 artemis teleport[1817446]: ERROR REPORT:
Mar 08 15:42:36 artemis teleport[1817446]: Original Error: *trace.BadParameterError client source and destination addresses should be valid same TCP version non-nil IP addresses
Mar 08 15:42:36 artemis teleport[1817446]: Stack Trace:
Mar 08 15:42:36 artemis teleport[1817446]:         github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:54 github.com/gravitational/teleport/lib/multiplexer.init
Mar 08 15:42:36 artemis teleport[1817446]:         runtime/proc.go:6735 runtime.doInit1
Mar 08 15:42:36 artemis teleport[1817446]:         runtime/proc.go:6702 runtime.doInit
Mar 08 15:42:36 artemis teleport[1817446]:         runtime/proc.go:249 runtime.main
Mar 08 15:42:36 artemis teleport[1817446]:         runtime/asm_arm64.s:1197 runtime.goexit
Mar 08 15:42:36 artemis teleport[1817446]: User Message: source address: [2600:3c06::f03d:91ff:fe73:1fc6]:23252, destination address: 192.168.64.254:443
Mar 08 15:42:36 artemis teleport[1817446]:         client source and destination addresses should be valid same TCP version non-nil IP addresses] pid:1817446.1 reversetunnel/srv.go:727

This appears to be related to the content of the X-Forwarded-For header and happens when proxy_service.trust_x_forwarded_for is set to true - which is needed to have accurate client source IPs in audit logs and for IP pinning.

What problem does this solve?

Now that Teleport supports pure websocket connection upgrades (introduced in v15.1.0 by #36343) it will become more common to use Cloudflare's free DNS proxying service to expose Teleport servers more securely. Cloudflare always serves IPv6 records on its free tier and this cannot be disabled due to an ideological stance on their end. It will still be more common than not to have this traffic directed to IPv4-only Teleport servers.

If a workaround exists, please include it.

Cloudflare provides a method called "Pseudo IPv4" which rewrites the X-Forwarded-For header to a Class E IPv4 address instead: https://developers.cloudflare.com/network/pseudo-ipv4/

Enabling the Pseudo IPv4 "Overwrite Headers" option under the "Network" settings for a Cloudflare domain causes Cloudflare to send the X-Forwarded-For address as a 250.x.x.x address instead of the actual IPv6 address. This does enable connectivity by meaning that the multiplexer won't reject IPv6 -> IPv4 PROXY protocol headers, but unfortunately makes the source IP for connections inaccurate.

@webvictim webvictim added feature-request Used for new features in Teleport, improvements to current should be #enhancements networking Network connectivity features/problems tls-routing Issues related to TLS routing ipv6 labels Mar 11, 2024
@pschisa pschisa added the c-dd Internal Customer Reference label Sep 3, 2024
@cmoiccool
Copy link

Are there any plans to implement this feature ?

I am hitting this problem when enabling trust_x_forwarded_for to true in a configuration using Nginx as a reverse proxy forwarding to the teleport server listening on a specific private IPv4 (not the 0.0.0.0 address). I wanted to try to work around this issue configuring an IPv6 link local address also as a web_listen_addr but it is not possible to setup multiple specific listen addresses in web_listen_addr.

@webvictim
Copy link
Contributor Author

For some reason Psuedo IPv4 is no longer working for me (maybe because I'm now using a Cloudflare tunnel for my traffic) so I had to disable all outbound IPv6 TCP connectivity to Cloudflare IP ranges from my own server to fix this. Not ideal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c-dd Internal Customer Reference feature-request Used for new features in Teleport, improvements to current should be #enhancements ipv6 networking Network connectivity features/problems tls-routing Issues related to TLS routing
Projects
None yet
Development

No branches or pull requests

4 participants