Reddit port 443 pitfall reproduction

[HouQiming]

English Only (except for bug reporting).

To reproduce the problem, you need 3 systems A, B, C and a remote https server D. Theoretically A and B could be the same system, but I was testing with them being different. They also play different roles so I'm separating them here for clarity.

The goal here is to create an udp2raw-obfuscated wireguard tunnel from A to D. A is the client. D is some random website controlled by a 3rd party. B is the udp2raw client. C is the udp2raw server. A and C are also wireguard peers. C has packet forwarding enabled.

There are multiple ethernet adapters and IP addresses of interest, here is a list for clarity:

System	Adapter	IP address	Purpose
A	eth0	IP_A_ETH	A's public-facing address for communicating with B
B	eth0	IP_B_ETH	B's public-facing address for A to connect to
C	eth0	IP_C_ETH	C's public-facing address for B to connect to
A	wg0	IP_A_WG	A's wireguard VPN address for tunneled communication with C
C	wg0	IP_C_WG	C's wireguard VPN address for tunneled communication with A
D	n/a	IP_D	D's public-facing server address which A intends to access through C
D	n/a	DOMAIN_D	D's domain name, 8.8.8.8 resolves it to ${IP_D} when queried from C but to a different value when queried from A

Since the IP addresses are sensitive, I won't reveal their values and would refer to them as environment variables.

The list of running services:

• On A, wireguard configured with peer ${IP_B_ETH}:${WG_PORT}
• On A, ip -4 route add ${IP_D}/32 via ${IP_A_WG} dev wg0 metric 10
• On B, ./udp2raw_amd64 -c -l ${IP_B_ETH}:${WG_PORT} -r ${IP_C_ETH}:443 -k ${PASSWD} --raw-mode faketcp -a
• On C, ./udp2raw_amd64 -s -l 0.0.0.0:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp -a
• On C, wireguard configured with listening port ${WG_PORT} and peer 127.0.0.1
• On C, IP forwarding enabled by echo 1 > /proc/sys/net/ipv4/ip_forward

C also has a firewall configured to block all incoming traffic except for TCP 22 / 443, so it's unlikely anything could connect by mistake. The wg0 adapaters on A and C have their MTU set to a conservative value 1000 to avoid size-related packet loss. All 3 systems have IPv6 disabled.

The symptom:

On A, wget https://${DOMAIN_D} fails in the connection stage. But ping ${DOMAIN_D} works and nslookup ${DOMAIN_D} returns ${IP_D}.

The diagnostics:

tcpdump -n -i eth0 src port 443 or dst port 443 on C. Since the actual packet log is sensitive, I'll try to sum up the important part:

src	dst	content
IP_C_ETH	IP_D	TCP SYN
IP_D	IP_C_ETH	TCP SYN-ACK
IP_A_WG	IP_D	TCP SYN-ACK-ACK
IP_A_WG	IP_D	HTTPS handshake
IP_D	IP_C_ETH	TCP SYN-ACK (retransmit)

System C is actually sending out TCP packets with source address set to IP_A_WG, putting the VPN LAN address in its IP header. Here it just breaks the TCP handshake, but a tor configuration could have been leaking A's public address, which would be much worse.

Maybe the solution is to restrict udp2raw to a particular adapter on the server side: it's very unlikely for any packet from wg0 to be intended for udp2raw.

[wangyu-]

This issue is a continuation of the below reddit thread:
https://www.reddit.com/r/WireGuard/comments/lxddyt/til_udp2raw_tunneling_pitfall/

[wangyu-]

I do have reproduced the problem. In my environment A and B is the same machine, tinyfecVPN is used instead of wireguard.

The problem is not caused by the iptables rule, it's caused by something else. I have done some test by mannually adding the ipables rule:

1. (udp2raw server) listen on 0.0.0.0 and iptables rule with 0.0.0.0

./udp2raw_amd64 -s -l 0.0.0.0:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp
iptables -I INPUT -d 0.0.0.0/0 -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 

this one has the problem

2. listen on ${IP_C_ETH} and iptables rule with ${IP_C_ETH}

./udp2raw_amd64 -s -l ${IP_C_ETH}:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp
iptables -I INPUT -d ${IP_C_ETH} -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 

this one has no problem

3. listen on ${IP_C_ETH} and iptables rule with 0.0.0.0

./udp2raw_amd64 -s -l ${IP_C_ETH}:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp
iptables -I INPUT -d 0.0.0.0/0 -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 

this one has no problem

4. listen on ${0.0.0.0} and iptables rule with ${IP_C_ETH}

./udp2raw_amd64 -s -l ${0.0.0.0}:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp
iptables -I INPUT -d ${IP_C_ETH} -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 

this one has the problem

So, according to the test, the iptables rule doesn't matter. What makes a difference is the address udp2raw server is listening on.

The reason is, when you let udp2raw listen on 0.0.0.0, udp2raw doesn't know the IP address of the interfaces on your machine. In the case 1 &4, when SYN comes from wg0 with dest ${IP_D}:443, udp2raw doesn't know the destination is another machine, udp2raw server assumes this packet is sending to him, so udp2raw server replied SYN-ACK before the really https server. In this way the TCP handshake is hijacked by udp2raw, wget has got the wrong sequence number from udp2raw, so it's not able to talk with the real https server anymore.
(when you let udp2raw listen on ${IP_C_ETH}, you told udp2raw the IP address of the interface, then udp2raw is able to know ${IP_D}:443 is not sending to him. Then there is no problem.)

Solutions

The short-term solution is, as the OP mentioned, when you run udp2raw on a common port (such as 443), don't let udp2raw listen on 0.0.0.0, let udp2raw listen on the specific IP instead.

The long-term solution is, I will add codes to let udp2raw automatically get the IP address for every interface (when listening on 0.0.0.0). Then udp2raw will be able to detect whether the packet is send to him, or it's a forwarding packet.

[HouQiming]

Thanks for the explanation! Learned a lot from it. Looking forward to the solution.

…

On Fri, Mar 12, 2021 at 4:48 AM wangyu- ***@***.***> wrote: I do have reproduced the problem. In my environment A and B is the same machine, tinyfecVPN is used instead of wireguard. The problem is not caused by the iptables rule, it's caused by something else. I have done some test by mannually adding the ipables rule: 1. (udp2raw server) listen on 0.0.0.0 and iptables rule with 0.0.0.0 ./udp2raw_amd64 -s -l 0.0.0.0:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp iptables -I INPUT -d 0.0.0.0/0 -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 this one has the problem 2. listen on ${IP_C_ETH} and iptables rule with ${IP_C_ETH} ./udp2raw_amd64 -s -l ${IP_C_ETH}:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp iptables -I INPUT -d ${IP_C_ETH} -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 this one has no problem 3. listen on ${IP_C_ETH} and iptables rule with 0.0.0.0 ./udp2raw_amd64 -s -l ${IP_C_ETH}:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp iptables -I INPUT -d 0.0.0.0/0 -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 this one has no problem 4. listen on ${0.0.0.0} and iptables rule with ${IP_C_ETH} ./udp2raw_amd64 -s -l ${0.0.0.0}:443 -r 127.0.0.1:${WG_PORT} -k ${PASSWD} --raw-mode faketcp iptables -I INPUT -d ${IP_C_ETH} -p tcp -m tcp --dport 443 -j udp2rawDwrW_28e59715_C0 this one has the problem So, according to the test, the iptables rule doesn't matter. What makes a difference is the address udp2raw server is listening on. The reason is, when you let udp2raw listen on 0.0.0.0, udp2raw doesn't know the IP address of the interfaces on your machine. In the case 1 &4, when SYN comes from wg0 with dest ${IP_D}:443, udp2raw doesn't know the destination is another machine, udp2raw server assumes this packet is sending to him, so udp2raw server replied SYN-ACK before the really https server. In this way the TCP handshake is hijacked by udp2raw, wget has got the wrong sequence number from udp2raw, so it's not able to talk with the real https server anymore. (when you let udp2raw listen on ${IP_C_ETH}, you told udp2raw the IP address of the interface, then udp2raw is able to know ${IP_D}:443 is not sending to him. Then there is no problem.) Solutions The short-term solution is, as the OP mentioned, when you run udp2raw on a common port (such as 443), don't let udp2raw listen on 0.0.0.0, let udp2raw listen on the specific IP instead. The long-term solution is, I will add codes to let udp2raw automatically get the IP address for every interface (when listening on 0.0.0.0). Then udp2raw will be able to detect whether the packet is send to him, or it's a forwarding packet. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/wangyu-/udp2raw-tunnel/issues/389#issuecomment-797038924>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5MQFYB7WZ7VZX6ZE664J3TDENCNANCNFSM4Y7PKKYQ> .