tuning for maximum performance

[totaam]

I am using libspng to compress my RGB(A) pixel data and after following the code and some empirical testing (mostly to validate those choices), I came up with these settings:

• SPNG_IMG_COMPRESSION_LEVEL=1
• SPNG_TEXT_WINDOW_BITS=15
• SPNG_IMG_COMPRESSION_STRATEGY=Z_HUFFMAN_ONLY
• SPNG_IMG_MEM_LEVEL=MAX_MEM_LEVEL
  You can see these settings in the cython encoder source here:
  https://github.com/Xpra-org/xpra/blob/76ae51d77dd477c5e53e4547ae84fecea7fc0e8b/xpra/codecs/spng/encoder.pyx#L175-L191

Unfortunately, when comparing the performance of this Cython module with Python-pillow, using this trivial test tool: Xpra-org/xpra@4fd50ba, I found that libspng compresses very consistently at around 4MPixels/s whereas libpng achieves 8 to 16MPixels/s on my i6700K.
What am I doing wrong?

(this is on Fedora 34 with libspng git master)

[randy408]

Filtering is still enabled which will slow things down, what options are passed to libpng?

[totaam]

Filtering is still enabled which will slow things down

Unless there's another option I have missed, it should be set to NONE:
https://github.com/Xpra-org/xpra/blob/76ae51d77dd477c5e53e4547ae84fecea7fc0e8b/xpra/codecs/spng/encoder.pyx#L167-170

    ihdr.compression_method = 0
    ihdr.filter_method = SPNG_FILTER_NONE
    ihdr.interlace_method = SPNG_INTERLACE_NONE

what options are passed to libpng?

Good question! I'm not exactly sure since it is used through python-pillow just by calling img.save(buf, pil_fmt, **kwargs) here:
https://github.com/Xpra-org/xpra/blob/76ae51d77dd477c5e53e4547ae84fecea7fc0e8b/xpra/codecs/pillow/encoder.py#L240
kwargs is empty since we construct images from raw RGB pixel data, so that would be using the default settings.

This is kind of one of the key reasons for wanting to use libspng instead: a better API with direct access from our Cython module. But we need to at least match the performance of libpng.

[randy408]

ihdr.filter_method = SPNG_FILTER_NONE

That enum is not meant for that field, the only valid value for filter_method is zero.

For filter method 0 there are 5 filter types, try reducing it to 2 with spng_set_option(ctx, SPNG_FILTER_CHOICE, SPNG_FILTER_CHOICE_NONE | SPNG_FILTER_CHOICE_SUB).

If the options don't match exactly then performance won't be comparable, not only filtering has its overhead but compression speed is affected by it.

[totaam]

That enum is not meant for that field

Oh! Good thing I asked!

For filter method..

Thanks for the hint!
Using SPNG_FILTER_CHOICE_SUB is still too slow for our use case but SPNG_FILTER_CHOICE_NONE is fast enough: on average 2 to 2.5 times faster than libpng with default settings.

If the options don't match exactly then performance won't be comparable

That's OK, I understand.
We need this for real-time compression, so the settings don't really matter all that much as long as the performance is improved without sacrificing output size.
And it does that: performance is now better by a good margin whilst the output size is similar enough.

Thank you for your help.