vaapi encoder on AMD exceeds DATA_SHARDS_MAX due to NO_RC_BUF_LIMIT

[CypherGrue]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the latest beta/pre-release?

This issue is present in the latest pre-release

Describe the Bug

NO_RC_BUF_LIMIT was added to Linux vaapi encoder configuration for Intel GPUs to improve stream quality #1255 9e23b39. This has caused issues on AMD hardware where the encoder is not configured with the necessary bufsize.

Expected Behavior

No response

Additional Context

No response

Host Operating System

Linux

Operating System Version

Ubuntu 24.04 Linux 6.9.3

Architecture

64 bit

Sunshine commit or version

master 90fd371

Package

Linux - AppImage

GPU Type

AMD

GPU Model

RX7600

GPU Driver/Mesa Version

Mesa 24.0.5

Capture Method

None

Config

encoder = vaapi

Apps

No response

Relevant log output

[2024:07:01:10:41:20]: Warning: Number of fragments for reed solomon exceeds DATA_SHARDS_MAX
267 payload 311799 > 255, skipping error correction

This issue is a fork from a related closed issue #1040, which contains relevant discussions.

[CypherGrue]

@ns6089 @istori1 @cgutman Rather than adding manufacturer strings into the code, might it be better to wait for a re-test of the current Intel drivers? Some of their GPUs are very new and there has been (and will continue to be) a lot of driver development over the year. Maybe NO_RC_BUF_LIMIT is no longer necessary at all, or maybe it's a driver bug that can be reported.

[goncalossilva]

I have the same issue on Fedora with a 6900XT. Firing a shotgun in Allan Wake II stutters completely. Removing NO_RC_BUF_LIMIT does help.

[peacey]

I also have the same issue with a 7900XTX with the same fragments error in the logs with stuttering. Removing NO_RC_BUF_LIMIT fixes the issue for me too.

[MrChrisDurban]

I also have the same issue with a 7900XTX with the same fragments error in the logs with stuttering. Removing NO_RC_BUF_LIMIT fixes the issue for me too.

Im seeing the same issue on 6950xt. Where exactly would i remove the "NO_RC_BUF_LIMIT"?

[goncalossilva]

See the change in #1255. That's the change you need to reverse to have sunshine working properly on AMD GPUs. Unfortunately you then have to (re-)compile sunshine, unless someone provides patched binaries.

[cgutman]

@CypherGrue please try this AppImage build from #2821 and confirm the issue is resolved: https://github.com/LizardByte/Sunshine/actions/runs/9831423949/artifacts/1675753274