Unexpected mouse behaviour in some situations with UE5

[netlore]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the latest beta/pre-release?

This issue is present in the latest pre-release

Describe the Bug

In some newer UE5 games, certain UI elements are either inaccessible with the mouse, or cause the mouse to warp to a different location on the screen when the mouse button is released.

Expected Behavior

Eg1:- In "Pax Dei (steam)" there are rotary style menus for some functions were a menu is represented by circular UI projected over the world (Hold down T or B to access one of the menus in the game). These menus simply do not respond to mouse clicks at all.
Without some of the options being accessible the game is not really playable, as you cannot recover your corpse if you die!

Eg2:- In "Ashes of Creation" Alpha 2, right or left clicking on the world (to steer or move the camera) or clicking on certain UI elements will result in the mouse warping to the top left corner of the screen as soon as the cursor becomes visible again. This means that, at best one is continually repositioning the mouse on the mouse mat which makes high pressure gameplay almost impossible, and at worst, one cannot use a right click pulldown (Eg. Leave Party) menu because the mouse instantly moves away from the menu and it disappears.

Additional Context

This is definitely dependent on something the engine is doing, but it does not happen when I am not streaming, so it must also be something that in a perfect world, Sunshine could resolve.

Note that the versions listed below are the very latest Sunshine pre-release, and Nvidia driver, because I upgraded them as far as possible as part of the troubleshooting process,... the version I originally used was Nvidia 560, and Sunshine from Oct 2023, but upgrading did not change the issue.

Also note, that at least for "Pax Dei" the problem appears to be Windows specific, since if I run the game on Linux (using steam/proton) and stream it from there using the Linux version of Sunshine, then there is no problem.... however I cannot run "Ashes of Creation" on Linux to test.... However my primary use case is to stream /From/ Windows for obvious reasons.

Host Operating System

Windows

Operating System Version

Windows 10 24H2 Build 19045.5011

Architecture

amd64/x86_64

Sunshine commit or version

v2024.1025.12635

Package

Windows - installer (recommended)

GPU Type

Nvidia

GPU Model

1080Ti

GPU Driver/Mesa Version

566.03

Capture Method

None

Config

sunshine_name = Gamesrv
controller = disabled
native_pen_touch = disabled

NB: I disabled controller and pen_touch as part of trying to debug the issue, but it made no difference.

Apps

No response

Relevant log output

See the logs uploaded by Sunshine
[2024-10-29 11:51:09.620]: Info: Sunshine version: v2024.1025.12635
[2024-10-29 11:51:09.620]: Info: Package Publisher: LizardByte
[2024-10-29 11:51:09.620]: Info: Publisher Website: https://app.lizardbyte.dev
[2024-10-29 11:51:09.620]: Info: Get support: https://app.lizardbyte.dev/support
[2024-10-29 11:51:09.857]: Info: nvprefs: No need to modify application profile settings
[2024-10-29 11:51:09.857]: Info: nvprefs: Changed OGL_CPL_PREFER_DXPRESENT to OGL_CPL_PREFER_DXPRESENT_PREFER_ENABLED for base profile
[2024-10-29 11:51:09.910]: Info: Compiling shaders...
[2024-10-29 11:51:10.014]: Warning: include/base_vs.hlsl(28,13-36): warning X3556: integer modulus may be much slower, try using uints if possible.
[2024-10-29 11:51:10.054]: Warning: include/base_vs.hlsl(28,13-36): warning X3556: integer modulus may be much slower, try using uints if possible.
[2024-10-29 11:51:10.087]: Warning: include/base_vs.hlsl(28,13-36): warning X3556: integer modulus may be much slower, try using uints if possible.
[2024-10-29 11:51:10.110]: Warning: include/base_vs.hlsl(28,13-36): warning X3556: integer modulus may be much slower, try using uints if possible.
[2024-10-29 11:51:10.174]: Warning: include/base_vs.hlsl(28,13-36): warning X3556: integer modulus may be much slower, try using uints if possible.
[2024-10-29 11:51:10.195]: Warning: include/base_vs.hlsl(28,13-36): warning X3556: integer modulus may be much slower, try using uints if possible.
[2024-10-29 11:51:10.195]: Info: Compiled shaders
[2024-10-29 11:51:10.252]: Info: Successfully reset default audio device
[2024-10-29 11:51:10.254]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024-10-29 11:51:10.254]: Info: Trying encoder [nvenc]
[2024-10-29 11:51:10.344]: Info: ddprobe.exe 1  --verify-frame-capture returned 0x887A0004
[2024-10-29 11:51:10.497]: Info: ddprobe.exe 2  --verify-frame-capture returned 0x00000000
[2024-10-29 11:51:10.497]: Info: Set GPU preference: 2
[2024-10-29 11:51:10.713]: Info: 
Device Description : NVIDIA GeForce GTX 1080 Ti
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001B06
Device Video Mem   : 11106 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16271 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024-10-29 11:51:10.713]: Info: Active GPU has HAGS disabled
[2024-10-29 11:51:10.713]: Info: Using realtime GPU priority
[2024-10-29 11:51:10.713]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 8
Red Primary        : [0.679688,0.30957]
Green Primary      : [0.206055,0.693359]
Blue Primary       : [0.151367,0.0546875]
White Point        : [0.313477,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024-10-29 11:51:10.714]: Info: Desktop resolution [1920x1080]
[2024-10-29 11:51:10.714]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024-10-29 11:51:10.714]: Info: Display refresh rate [59.885Hz]
[2024-10-29 11:51:10.714]: Info: Requested frame rate [60fps]
[2024-10-29 11:51:10.714]: Info: Creating encoder [h264_nvenc]
[2024-10-29 11:51:10.714]: Info: Color coding: SDR (Rec. 601)
[2024-10-29 11:51:10.714]: Info: Color depth: 8-bit
[2024-10-29 11:51:10.714]: Info: Color range: JPEG
[2024-10-29 11:51:10.771]: Info: NvEnc: created encoder H.264 P1 two-pass
[2024-10-29 11:51:10.800]: Info: Creating encoder [hevc_nvenc]
[2024-10-29 11:51:10.800]: Info: Color coding: SDR (Rec. 601)
[2024-10-29 11:51:10.800]: Info: Color depth: 8-bit
[2024-10-29 11:51:10.800]: Info: Color range: JPEG
[2024-10-29 11:51:10.860]: Info: NvEnc: created encoder HEVC P1 two-pass
[2024-10-29 11:51:10.888]: Info: Creating encoder [av1_nvenc]
[2024-10-29 11:51:10.888]: Info: Color coding: SDR (Rec. 601)
[2024-10-29 11:51:10.888]: Info: Color depth: 8-bit
[2024-10-29 11:51:10.888]: Info: Color range: JPEG
[2024-10-29 11:51:10.905]: Error: NvEnc: encoding format is not supported by the gpu
[2024-10-29 11:51:10.911]: Info: Creating encoder [h264_nvenc]
[2024-10-29 11:51:10.911]: Info: Color coding: SDR (Rec. 601)
[2024-10-29 11:51:10.911]: Info: Color depth: 8-bit
[2024-10-29 11:51:10.911]: Info: Color range: JPEG
[2024-10-29 11:51:10.964]: Info: NvEnc: created encoder H.264 P1 yuv444 two-pass
[2024-10-29 11:51:11.200]: Info: 
Device Description : NVIDIA GeForce GTX 1080 Ti
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001B06
Device Video Mem   : 11106 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 16271 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024-10-29 11:51:11.200]: Info: Active GPU has HAGS disabled
[2024-10-29 11:51:11.200]: Info: Using realtime GPU priority
[2024-10-29 11:51:11.200]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 8
Red Primary        : [0.679688,0.30957]
Green Primary      : [0.206055,0.693359]
Blue Primary       : [0.151367,0.0546875]
White Point        : [0.313477,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024-10-29 11:51:11.200]: Info: Desktop resolution [1920x1080]
[2024-10-29 11:51:11.200]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024-10-29 11:51:11.200]: Info: Display refresh rate [59.885Hz]
[2024-10-29 11:51:11.200]: Info: Requested frame rate [60fps]
[2024-10-29 11:51:11.201]: Info: Creating encoder [hevc_nvenc]
[2024-10-29 11:51:11.201]: Info: Color coding: SDR (Rec. 709)
[2024-10-29 11:51:11.201]: Info: Color depth: 10-bit
[2024-10-29 11:51:11.201]: Info: Color range: JPEG
[2024-10-29 11:51:11.419]: Info: NvEnc: created encoder HEVC P1 yuv444 10-bit two-pass
[2024-10-29 11:51:11.494]: Info: 
[2024-10-29 11:51:11.494]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024-10-29 11:51:11.494]: Info: 
[2024-10-29 11:51:11.494]: Info: Found H.264 encoder: h264_nvenc [nvenc]
[2024-10-29 11:51:11.494]: Info: Found HEVC encoder: hevc_nvenc [nvenc]
[2024-10-29 11:51:11.530]: Info: Configuration UI available at [https://localhost:47990]
[2024-10-29 11:51:12.281]: Info: Registered Sunshine mDNS service

[netlore]

Emm, I cannot test with Ashes of Creation until the test server is back up, but I found a solution that works with Pax Dei.... "plug a mouse in". This computer sits in a server rack, and has only power and network (thought it has a dummy HDMI plug to get the display to initialise)... but it has no keyboard or mouse plugged in normally. But while testing I went to the console to test something and left the kvm switched to it... whereupon I discovered that connecting a mouse to the machine solved the problem at least with Pax Dei.

I'll report at the weekend whether this solves the problem with Ashes of Creation Alpha 2, but it certainly points towards the cause of this issue, and perhaps it's something that can be resolved in the software, or at least noted in the documentation.

[ReenigneArcher]

We can certainly add a note to the docs about this. I'm not sure if there's anything we can do on the software side or not.

[netlore]

I've confirmed that plugging in a physical mouse did solve the issue with Ashes of Creation as well, so I think the issue can be closed if it cannot be solved in software... I see the documentation update and that's good enough for me, it's good that people will have some hint that this might be an issue.

I have also observed that the same issue occurs with Steam streaming also!