[Q] interpolation and --desktop-scaling, sound

[Lanchon]

hi,

i am using xpra v5.0.5-r1 in shadow mode (both server and client are debian base). the server desktop is too large for the client so i am using --desktop-scaling=0.75 in the client command.

1. image interpolation used is zero-order (the null interpolation, aka nearest neighbor, aka pixel/line discarding) which makes small text really hard to read. curiously, the first frame seems to use linear interpolation but anything after that does not, which maybe points to some kind of adaptive decision going on. how does one force an image interpolation method (regardless of resulting frame rate)?

2. when using --desktop-scaling=0.75 in the client command, who does the scaling/interpolation? server or client?

3. the server is linux mint (ubuntu 22.04 base). after a server reboot, only the first time i connect i get the remote server audio in the client as expected, whereas audio is not forwarded in following connections. (i did not try just logging out instead of rebooting.) any pointers to where i should look for more info about this?

thanks for xpra!

[totaam]

the server desktop is too large for the client so i am using --desktop-scaling=0.75 in the client command

We should be able to auto-scale it on full-screen / maximization - there are existing tickets for this.
Work in progress, but not for this release.

curiously, the first frame seems to use linear interpolation

The first frame is probably done client-side because the server sends it early, before it realizes that downscaling can be done server side.

how does one force an image interpolation method (regardless of resulting frame rate)?

You can't at the moment.
And it varies from distro to distro, depending on what scaling modules are available and also what rendering backend is used (--opengl=no|force)

when using --desktop-scaling=0.75 in the client command, who does the scaling/interpolation? server or client?

The server, that's the only way to save bandwidth by not sending extra pixels:
#2052

only the first time i connect i get the remote server audio in the client as expected, whereas audio is not forwarded in following connections

Please file a separate issue - that should not happen, audio runs in a separate process so you should be able to start and stop it as many times are you like without interferring with anything.

[Lanchon]

@totaam

thank you!

We should be able to auto-scale it on full-screen / maximization - there are existing tickets for this.

i'll look for those and add my $0.02 there.

curiously, the first frame seems to use linear interpolation

The first frame is probably done client-side because the server sends it early, before it realizes that downscaling can be done server side.

hmm... the server bitstream must encode whether scaling has been done then. i suppose i can't disable server scaling with a client argument, can i? this would seem to force the client to scale it instead. (i know, it's inefficient! but better inefficient than unreadable.)

[interpolation] varies from distro to distro, depending on what scaling modules are available and also what rendering backend is used (--opengl=no|force)

server runs on a ubuntu 22.04 (jammy) base. if there are packages i can add to add scaling modules, let me know.

my problem is that i started using xpra to help some friends, but i don't want to bother them (too much) to make tests. they just want to do simple stuff on their PCs, they are mostly non-tech. i should run a mock setup to investigate further instead of asking so many questions.

thanks!

[totaam]

Scaling questions now in: #4133

the server bitstream

bitstream is a loaded term, we don't have a bitstream per-se, just screen updates, some of which may use video bitstreams.

must encode whether scaling has been done then

Yes, that can be seen using the -d compress command line option.

i suppose i can't disable server scaling with a client argument, can i?

--video-scaling=0 should disable scaling unless absolutely necessary.
For example, some clients have a limit on window size, or OpenGL rendering size.
4k was quite a common limit - nowadays, it's more likely to be 16k.
The html5 client has a lower limit for performance:
https://github.com/Xpra-org/xpra-html5/blob/1789bb05cf96ba6f864c9a50f0eb4458ee0c013e/html5/js/Client.js#L160-L163
Higher with the native offscreen decoder.

i know, it's inefficient! but better inefficient than unreadable

The quality should be pretty much the same.
The server will use libyuv or swscale to downscale.
For libyuv, the filer mode depends on the current speed setting:

xpra/xpra/codecs/libyuv/converter.pyx

Lines 105 to 110 in c138980

	cdef inline FilterMode get_filtermode(int speed):
	if speed>66:
	return kFilterNone
	elif speed>33:
	return kFilterBilinear
	return kFilterBox

When using one of the nvidia modules, it will use a CUDA kernel to do the scaling, if required - those CUDA kernels only support nearest neighbour at the moment.
The client will use either the same software modules (for software rendering via cairo) or OpenGL upscaling when opengl is enabled. This uses OpenGL texture sampling in the shaders:
https://github.com/Xpra-org/xpra/blob/c1389802d3169275b64c00361178a03fd3a46e90/xpra/client/gl/shaders.py

varies from distro to distro, depending on what scaling modules are available and also what rendering backend is used (--opengl=no|force)

server runs on a ubuntu 22.04 (jammy) base. if there are packages i can add to add scaling modules, let me know.

A default server installation should have the software scaling modules - it may or may not have support for the nvidia modules. It does not need anything OpenGL related at all.
A default client installation should have all the modules required to support OpenGL accelerated rendering.