AVIF encoding

[TijZwa]

We might consider using AVIF (https://en.wikipedia.org/wiki/AVIF) for encoding

Possible starting point: https://github.com/cgohlke/imagecodecs

[totaam]

Feature support: https://caniuse.com/avif

Of the major browsers, Firefox and Chrome have it, Edge and Safari do not.

[totaam]

Adding the decoding to the html5 client was trivial: Xpra-org/xpra-html5@29ffa3f, but the encoding side is more tricky.
The biggest problem is the performance:

compress: 1384.4ms for  499x316  pixels at    0,0    for wid=3     \
    using      avif with ratio   0.9%  (  615KB to     5KB), \
    sequence     1, client_options={'encoder': 'avif'}, \
    options={'quality': 100, 'speed': 80, \
        'rgb_formats': ('RGBX', 'RGBA', 'RGB'), \
    'zlib': True, 'lz4': True, 'content-type': 'text', \
    'scroll': True, 'cuda-device-context': None}

That's an xterm, which normally takes a few milliseconds with all the other encoders!

[totaam]

MacOS packaging required some work: Xpra-org/gtk-osx-build#23

[totaam]

Tested and working with the commits above and the python client, also with the html5 client.

Issues remaining:

• selecting a backend - they probably have vastly different performance characteristics, but we only have a single codec interface... tough
• subsampling, quality, lossless mode?
• color profile
• use decoder->alphaPresent? alphaPremultiplied?
• decode to YUV for opengl backend
• use avif whenever appropriate (needs benchmarking with corpus of images)
• high bit depth support

[totaam]

Apart from high bit depth support - which would be reasonably easy to add with a conversion step, all the issues have been addressed.

The problem is that the performance just isn't good enough and I suspect that we will have the same issue with AV1: #1668.

Typical example:

some.png : <PIL.Image.Image image mode=RGB size=920x832 at 0x7F5CF6C5DC40>
rgb24      rgb          :      106772.2 MPixels/s    size=  224250KB    100.0%
rgb32      rgb          :      148436.4 MPixels/s    size=  224250KB    100.0%
png        pillow       :          30.4 MPixels/s    size=   18226KB    8.1%
png/L      pillow       :          65.9 MPixels/s    size=    9835KB    4.4%
png/P      pillow       :          42.0 MPixels/s    size=   15571KB    6.9%
jpeg       pillow       :         298.0 MPixels/s    size=    8217KB    3.7%
webp       pillow       :          44.8 MPixels/s    size=    7701KB    3.4%
png        spng         :          73.1 MPixels/s    size=   20584KB    9.2%
png/L      spng         :          79.8 MPixels/s    size=   25178KB    2.5%
webp       webp         :          64.3 MPixels/s    size=    5287KB    2.4%
jpeg       jpeg         :         363.5 MPixels/s    size=   11215KB    5.0%
jpega      jpeg         :         356.3 MPixels/s    size=   11215KB    5.0%
avif       avif         :           3.3 MPixels/s    size=   18917KB    8.4%

avif on its fastest setting (which we use by default - because all the other settings are too slow) and at only 50% quality compresses less than the other encoders (level with png - which is lossless!) and it is much slower: 10 times slower or more.

So it is included and enabled by default, but we won't be choosing it if jpeg or webp or png are available.