TCP tunnel opens and disconnects again on MacOS

[premun]

It seems like that for Apple devices, it can happen that:

• We create the TCP tunnel in MacOS
• We start the app
• The TCP tunnel disconnects again
• App fails to connect and runs without it, logs app end tag in its stdout
• XHarness sees the end tag, which sort of mean success, but also never received testResults.xml over the broken TCP, which means failure
• Together, it qualifies as APP_CRASH as the app failed to connect and send stuff but only because XHarness cannot tell the TCP connection broke

Example log:

[14:51:49] dbug: Starting TCP tunnel between mac port: 58234 and device port 58234
[14:51:49] dbug: [TCP tunnel]
[14:51:49] dbug: [TCP tunnel] Running /private/tmp/helix/working/A42E0919/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22375.5/tools/net7.0/any/../../../runtimes/any/native/mlaunch/bin/mlaunch --sdkroot /Applications/Xcode13.app --tcp-tunnel=58234:58234 --devname bd7aed3884d86af1ca7c2311e8706b5a95f3a300 -v -v -v -v -v
[14:51:49] dbug: [TCP tunnel] Using Xcode 13.0 found in /Applications/Xcode13.app
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Device discovery started
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Device discovery event: Connected (bd7aed3884d86af1ca7c2311e8706b5a95f3a300)
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Connected to DNCENGTVOS-071 (bd7aed3884d86af1ca7c2311e8706b5a95f3a300) in 00:00:00.0020177
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Creating USB tunnel between the port 58234 on the device and the port 58234 on the mac.
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
...
[14:51:54] dbug: [TCP tunnel] Please connect the device 'Tcp tunnel started on device port 58234 to host port 58234 for device bd7aed3884d86af1ca7c2311e8706b5a95f3a300'...
[14:51:54] dbug: TCP tunnel created on port 58234
[14:51:54] dbug: Starting the application
...
[14:51:58] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:51:58] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:51:58] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:51:58] dbug: Detected test end tag in application's output
[14:51:58] dbug: Process mlaunch exited with 137
[14:51:58] dbug: Test run completed
...
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:52:03] dbug: Killing process tree of 64383...
[14:52:03] dbug: Pids to kill: 64383
[14:52:03] dbug: [TCP tunnel] Killing process 64389 as it was cancelled
[14:52:03] dbug: [TCP tunnel] Process mlaunch exited with 137
[14:52:03] dbug: Device log captured in /tmp/helix/working/A42E0919/w/A1AD0929/uploads/device-DNCENGTVOS-071-20220806_145148.log
[14:52:03] dbug: Test run started but crashed and no test results were reported
...
XHarness exit code: 80 (APP_CRASH)

Full log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-8c0cc100331f483699/System.Console.Manual.Tests/1/console.75d6ad8c.log?%3Fhelixlogtype%3Dresult

[premun]

Explanation linking into the log:

[premun]

Similar issue here which ended up as a timeout (wrongfully):

[02:53:39] dbug: [TCP tunnel]
[02:53:39] dbug: [TCP tunnel] Running /private/tmp/helix/working/93BF083D/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22451.1/tools/net7.0/any/../../../runtimes/any/native/mlaunch/bin/mlaunch --sdkroot /Applications/Xcode13.app --tcp-tunnel=51308:51308 --devname 986112a772e4ec5124df5adbd7e7d8d04c5efc1d -v -v -v -v -v
[02:53:40] dbug: [TCP tunnel] Using Xcode 13.0 found in /Applications/Xcode13.app
[02:53:40] dbug: [TCP tunnel] Xamarin.Hosting: Device discovery started
[02:53:40] dbug: [TCP tunnel] Xamarin.Hosting: Device discovery event: Connected (fff44:c6:5d:6d:09:3afff)
[02:53:40] dbug: [TCP tunnel] 2022-09-08 02:53:40.315 mlaunch[42597:707763]
[02:53:40] dbug: [TCP tunnel] Unhandled Exception:
[02:53:40] dbug: [TCP tunnel] Xamarin.Hosting.MobileDeviceException: Read Error (error: 0xe8000004)
[02:53:40] dbug: [TCP tunnel]   at Xamarin.Hosting.RealDevice.Connect () [0x00021] in /Users/builder/azdo/_work/3/s/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting/RealDevice.cs:57
[02:53:40] dbug: [TCP tunnel]   at Xamarin.Hosting.RealDevice..ctor (System.IntPtr am_device) [0x00022] in /Users/builder/azdo/_work/3/s/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting/RealDevice.cs:40
[02:53:40] dbug: [TCP tunnel]   at Xamarin.Hosting.DeviceDiscovery+<>c__DisplayClass23_1.<DiscoverNotificationCallback>b__0 () [0x00007] in /Users/builder/azdo/_work/3/s/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting/DeviceDiscovery.cs:225
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00014] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/thread.cs:74
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:968
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:910
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x0002b] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:899
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ThreadHelper.ThreadStart () [0x00008] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/thread.cs:111
[02:53:40] dbug: [TCP tunnel] [ERROR] FATAL UNHANDLED EXCEPTION: Xamarin.Hosting.MobileDeviceException: Read Error (error: 0xe8000004)
[02:53:40] dbug: [TCP tunnel]   at Xamarin.Hosting.RealDevice.Connect () [0x00021] in /Users/builder/azdo/_work/3/s/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting/RealDevice.cs:57
[02:53:40] dbug: [TCP tunnel]   at Xamarin.Hosting.RealDevice..ctor (System.IntPtr am_device) [0x00022] in /Users/builder/azdo/_work/3/s/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting/RealDevice.cs:40
[02:53:40] dbug: [TCP tunnel]   at Xamarin.Hosting.DeviceDiscovery+<>c__DisplayClass23_1.<DiscoverNotificationCallback>b__0 () [0x00007] in /Users/builder/azdo/_work/3/s/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting/DeviceDiscovery.cs:225
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00014] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/thread.cs:74
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:968
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:910
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x0002b] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:899
[02:53:40] dbug: [TCP tunnel]   at System.Threading.ThreadHelper.ThreadStart () [0x00008] in /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/src/Xamarin.Mac/mcs/class/referencesource/mscorlib/system/threading/thread.cs:111
[02:53:40] dbug: [TCP tunnel] Process mlaunch exited with 1
[02:53:40] dbug: SimpleTcpListener: an exception occurred in processing thread: System.InvalidOperationException: Tcp tunnel could not be initialized.
                    at Microsoft.DotNet.XHarness.iOS.Shared.Listeners.SimpleTcpListener.StartTcpTunnel() in /_/src/Microsoft.DotNet.XHarness.iOS.Shared/Listeners/SimpleTcpListener.cs:line 126
                    at Microsoft.DotNet.XHarness.iOS.Shared.Listeners.SimpleTcpListener.Start() in /_/src/Microsoft.DotNet.XHarness.iOS.Shared/Listeners/SimpleTcpListener.cs:line 209
                    at Microsoft.DotNet.XHarness.iOS.Shared.Listeners.SimpleListener.<StartAsync>b__19_0() in /_/src/Microsoft.DotNet.XHarness.iOS.Shared/Listeners/SimpleListener.cs:line 82