Unable to connect to Windows desktop (FIPS)

[bl-nero]

Expected behavior:
Seeing a Windows desktop when connecting through Web RDP.

Current behavior:

Bug details:

• Teleport version: v15.0.0-alpha.5, FIPS build (AWS AMI)
• Set up a FIPS cluster, turn on Windows desktop service on the same node
• Create a Lightsail Windows VM (Windows Server 2022), add to the desktop service according to the manual
• Attempt to connect using the Web UI.

Debug logs:

Jan 19 14:15:10 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:10Z DEBU [AUTH:1]    Server certificate cert(6f36078b-ce2f-4a6a-9ee2-e8420ac21762.fips1.bartosz.teleportdemo.net issued by fips1.bartosz.teleportdemo.net:304096904540262351967532334546212896831). auth/middleware.go:343
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WEB]       New desktop access websocket connection cluster-name:fips1.bartosz.teleportdemo.net desktop-name:bartosz-windows1 session:56b4 user:bl-nero web/desktop.go:74
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WEB]       "Attempting to connect to desktop using username=Administrator\n" cluster-name:fips1.bartosz.teleportdemo.net desktop-name:bartosz-windows1 session:56b4 user:bl-nero web/desktop.go:118
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WEB]       "Received screen spec: &{1720 1288}\n" cluster-name:fips1.bartosz.teleportdemo.net desktop-name:bartosz-windows1 session:56b4 user:bl-nero web/desktop.go:130
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU             generated user key for [bl-nero ec2-user -teleport-internal-join] with expiry on (1705781540) 2024-01-20 20:12:20.00118211 +0000 UTC keygen/keygen.go:165
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [AUTH]      Failed setting default kubernetes cluster for user login (user did not provide a cluster); leaving KubernetesCluster extension in the TLS certificate empty auth/auth.go:2607
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z INFO [CA]        Generating TLS certificate 1.3.9999.1.15=#13046e6f6e65,1.3.9999.1.9=#130d34362e32322e3136362e313331,1.3.9999.1.7=#131e66697073312e626172746f737a2e74656c65706f727464656d6f2e6e6574,CN=bl-nero,OU=usage:windows_desktop,O=access+O=editor+O=windows-desktop-admins,POSTALCODE={\"github_teams\":[\"dummy-team\"]\,\"kubernetes_groups\":null\,\"kubernetes_users\":null\,\"logins\":[\"bl-nero\"]},STREET=fips1.bartosz.teleportdemo.net,L=bl-nero+L=ec2-user+L=-teleport-internal-join dns_names:[] key_usage:5 not_after:2024-01-20 20:12:20.003570016 +0000 UTC tlsca/ca.go:1187
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z INFO [AUDIT]     cert.create cert_type:user cluster_name:fips1.bartosz.teleportdemo.net code:TC000I ei:0 event:cert.create client_ip:46.22.166.131 expires:2024-01-20T20:12:20.003570016Z logins:[bl-nero ec2-user -teleport-internal-join] prev_identity_expires:0001-01-01T00:00:00Z private_key_policy:none roles:[access editor windows-desktop-admins] route_to_cluster:fips1.bartosz.teleportdemo.net teleport_cluster:fips1.bartosz.teleportdemo.net github_teams:[dummy-team] kubernetes_groups:<nil> kubernetes_users:<nil> logins:[bl-nero] usage:[usage:windows_desktop] user:bl-nero time:2024-01-19T14:15:11.083Z uid:81e451b0-8350-466c-a46f-276eebdf2bd0 events/emitter.go:278
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [PROXY:SER] Dialing from: "46.22.166.131:37024" to: "ip-172-31-28-194.eu-central-1.compute.internal:3028". trace.fields:map[cluster:fips1.bartosz.teleportdemo.net] reversetunnel/localsite.go:308
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [HTTP:PROX] No proxy set in environment, returning direct dialer. proxy/proxy.go:197
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [PROXY:SER] Succeeded dialing from: "46.22.166.131:37024" to: "ip-172-31-28-194.eu-central-1.compute.internal:3028". trace.fields:map[cluster:fips1.bartosz.teleportdemo.net] reversetunnel/localsite.go:314
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WINDOWS_D] Ignoring unsupported cluster name "bartosz-windows1.desktop.teleport.cluster.local". pid:68400.1 service/desktop.go:186
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WEB]       Connected to windows_desktop_service cluster-name:fips1.bartosz.teleportdemo.net desktop-name:bartosz-windows1 session:56b4 user:bl-nero windows-service-addr:ip-172-31-28-194.eu-central-1.compute.internal:3028 windows-service-uuid:6f36078b-ce2f-4a6a-9ee2-e8420ac21762 web/desktop.go:188
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WINDOWS_D] Authenticated Windows desktop connection client-ip:46.22.166.131 pid:68400.1 desktop/windows_server.go:726
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WINDOWS_D] Connecting to Windows desktop client-ip:46.22.166.131 desktop-addr:172.26.1.113:3389 desktop-name:bartosz-windows1 pid:68400.1 desktop/windows_server.go:754
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WINDOWS_D] Got RDP username "Administrator" client-ip:46.22.166.131 desktop-addr:172.26.1.113:3389 desktop-name:bartosz-windows1 pid:68400.1 rdp-addr:172.26.1.113:3389 rdpclient/client.go:226
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WINDOWS_D] Got RDP screen size 1720x1288 client-ip:46.22.166.131 desktop-addr:172.26.1.113:3389 desktop-name:bartosz-windows1 pid:68400.1 rdp-addr:172.26.1.113:3389 rdpclient/client.go:246
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z DEBU [WINDOWS_D] Creating tracker for session acf8ceaa-34d0-4fc3-a80d-e6303a5291b6 pid:68400.1 desktop/windows_server.go:1278
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z INFO [AUDIT]     windows.desktop.session.start addr.local:46.22.166.131 addr.remote:172.26.1.113:3389 allow_user_creation:false cluster_name:fips1.bartosz.teleportdemo.net code:TDP00I desktop_addr:172.26.1.113:3389 teleport.dev/ad:false teleport.dev/origin:config-file desktop_name:bartosz-windows1 ei:1 event:windows.desktop.session.start login:Administrator private_key_policy:none proto:tdp sid:acf8ceaa-34d0-4fc3-a80d-e6303a5291b6 success:true time:2024-01-19T14:15:11.1Z uid:8298481b-d2a7-4560-9c0c-499698df635d user:bl-nero user_kind:1 windows_desktop_service:6f36078b-ce2f-4a6a-9ee2-e8420ac21762 windows_domain: windows_user:Administrator events/emitter.go:278
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z INFO [WINDOWS_D] Rust RDP loop starting client-ip:46.22.166.131 desktop-addr:172.26.1.113:3389 desktop-name:bartosz-windows1 pid:68400.1 rdp-addr:172.26.1.113:3389 rdpclient/client.go:264
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z INFO [WINDOWS_D] TDP input streaming starting client-ip:46.22.166.131 desktop-addr:172.26.1.113:3389 desktop-name:bartosz-windows1 pid:68400.1 rdp-addr:172.26.1.113:3389 rdpclient/client.go:329
Jan 19 14:15:11 ip-172-31-28-194.eu-central-1.compute.internal teleport[68400]: 2024-01-19T14:15:11Z INFO [CA]        Generating TLS certificate CN=Administrator dns_names:[] key_usage:1 not_after:2024-01-19 14:20:11.255888435 +0000 UTC tlsca/ca.go:1187

One time, before I enabled the debug logs, I also saw this in the logs:

Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: 2024-01-19T14:06:19Z INFO [WINDOWS_D] TDP input streaming finished client-ip:46.22.166.131 desktop-addr:172.26.1.113:3389 desktop-name:bartosz-windows1 pid:61804.1 rdp-addr:172.26.1.113:3389 rdpclient/client.go:343
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: [2024-01-19T14:06:19Z WARN  rdp_client] call_function_on_handle failed: handle not found
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: 2024-01-19T14:06:19Z INFO [AUDIT]     windows.desktop.session.end cluster_name:fips1.bartosz.teleportdemo.net code:TDP01I desktop_addr:172.26.1.113:3389 teleport.dev/ad:false teleport.dev/origin:config-file desktop_name:bartosz-windows1 ei:2 event:windows.desktop.session.end login:Administrator participants:[bl-nero] private_key_policy:none recorded:true session_start:2024-01-19T14:05:41.703Z session_stop:2024-01-19T14:06:19.517038302Z sid:bec9e61c-42cd-4488-bca3-1375099163de time:2024-01-19T14:06:19.517Z uid:f9190f22-3bbe-4332-a3df-b995c513d402 user:bl-nero user_kind:1 windows_desktop_service:6f36078b-ce2f-4a6a-9ee2-e8420ac21762 windows_domain: windows_user:Administrator events/emitter.go:278
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: 2024-01-19T14:06:19Z ERRO [WINDOWS_D] RDP connection failed: client_stop failed: 1 client-ip:46.22.166.131 desktop-addr:172.26.1.113:3389 desktop-name:bartosz-windows1 pid:61804.1 desktop/windows_server.go:758
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: 2024-01-19T14:06:19Z WARN [WINDOWS_D] could not record desktop recording event error:[
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: ERROR REPORT:
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: Original Error: *trace.ConnectionProblemError context canceled or timed out
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: Stack Trace:
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         github.com/gravitational/teleport/lib/events/session_writer.go:303 github.com/gravitational/teleport/lib/events.(*SessionWriter).RecordEvent
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         github.com/gravitational/teleport/lib/events/setter.go:167 github.com/gravitational/teleport/lib/events.SetupAndRecordEvent
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         github.com/gravitational/teleport/lib/srv/desktop/windows_server.go:967 github.com/gravitational/teleport/lib/srv/desktop.(*WindowsService).connectRDP.(*WindowsService).makeTDPSendHandler.func5
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         github.com/gravitational/teleport/lib/srv/desktop/tdp/conn.go:108 github.com/gravitational/teleport/lib/srv/desktop/tdp.(*Conn).WriteMessage
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         github.com/gravitational/teleport/lib/srv/desktop/tdp/conn.go:131 github.com/gravitational/teleport/lib/srv/desktop/tdp.(*Conn).SendNotification
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         github.com/gravitational/teleport/lib/srv/desktop/windows_server.go:690 github.com/gravitational/teleport/lib/srv/desktop.(*WindowsService).handleConnection.func1
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         github.com/gravitational/teleport/lib/srv/desktop/windows_server.go:763 github.com/gravitational/teleport/lib/srv/desktop.(*WindowsService).handleConnection
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]:         runtime/asm_amd64.s:1650 runtime.goexit
Jan 19 14:06:19 ip-172-31-28-194.eu-central-1.compute.internal teleport[61804]: User Message: context canceled or timed out] pid:61804.1 desktop/windows_server.go:968

[zmb3]

Please verify that the same instance works with a regular (non-FIPS) build before we put too much time into investigating this.

The error message looks like a timeout, which seems unrelated to FIPS.

[bl-nero]

@zmb3 I have just recreated a similar configuration on a non-FIPS v15 cluster; in a non-FIPS cluster, I'm able to connect to the instance. (Though this time, I see some screen artifacts, just as if the client couldn't decide on the desktop size. I'll file this as a separate issue.)

[bl-nero]

@zmb3 I have just checked with 14.3.3 (also FIPS) and it looks like there must be something inherently wrong in the way I set up these tests, because I also can't connect to an identical Windows machine (only this time the error message is different: "RDP connection failed").

[zmb3]

@ibeckermayer is able to connect with his FIPS build, so that would suggest that you have some sort of misconfiguration.

We will close this out after Isaiah confirms with a release candidate build.

[ibeckermayer]

Confirmed with Teleport Enterprise v15.0.0-beta.1 git:v15.0.0-beta.1-0-ga41de2b go1.21.6 X:boringcrypto