Improve quality of images

[umputun]

Looks like resized images with text in are very hard to read, super blurry. It is unclear what the reason - could be non-proportional ratio, could be the resizing method. Github, for instance, makes images more readable.

This is an example of the blurry one: https://remark42.radio-t.com/api/v1/picture/github_ef0f706a79cc24b17bbbb374cd234a691d034128/bnmk3aqhajfjbc6s020g.png

[akosourov]

Ok, I will take a look and try to understand how it could be improved

[mullakhmetov]

@akosourov are you working on this in any way? I am quite interested and want to solve it

[akosourov]

@mullakhmetov Yeah, sure. I tried to solve it two weeks ago but stopped and planned to continue on weekends. Ok, I don't mind :)

[mullakhmetov]

Current resize implementation uses draw.BiLinear interpolation. I've compare it with draw.CatmullRom (described as slow but high quality) and Lanczos3 (wiki).

Quality comparison

Text image

source	image/draw : BiLinear	image/draw : CatmullRom	resize :: Lanczos3

Cat image

source	image/draw : BiLinear	image/draw : CatmullRom	resize :: Lanczos3

Green image

source	image/draw : BiLinear	image/draw : CatmullRom	resize :: Lanczos3

I see the difference between BiLinear and CatmullRom, and don't see any advantages of Lanczos3 over CatmullRom.

Size comparison

Text image

Source image size: 673KB
Source image dimensions: 1125x2436
Dest image dimensions: 138x300

Algorithm	dest size
BiLinear	21KB
CatmullRom	23KB
Lanczos3	28KB

Cat image

Source image size: 817KB
Source image dimensions: 756x1008
Dest image dimensions: 143x190

Algorithm	dest size
BiLinear	41KB
CatmullRom	44KB
Lanczos3	45KB

Green image

Source image size: 1.6MB
Source image dimensions: 756x1008
Dest image dimensions: 143x190

Algorithm	dest size
BiLinear	59KB
CatmullRom	63KB
Lanczos3	64KB

Benchmarks

About the CatmullRom slowness. CatmullRom vs BiLinear bench results on my early 2015 macbook pro:

Text image

BenchmarkCatmullRom-4           1000000000               0.151 ns/op
BenchmarkBiLinear-4             1000000000               0.0980 ns/op

Cat image

BenchmarkCatmullRom-4           1000000000               0.0657 ns/op
BenchmarkBiLinear-4             1000000000               0.0482 ns/op

Green image

BenchmarkCatmullRom-4           1000000000               0.0781 ns/op
BenchmarkBiLinear-4             1000000000               0.0638 ns/op

Conclusion

So, due to clear quality advantages of CatmullRom over BiLinear and minor (in the web world) performance degradation I suggest to switch to CatmullRom.

@paskal @umputun what do you think?

[paskal]

CatmullRom is better than BiLinear, Lanczos3 looks a little better than both, but text image quality after resize is still awful. Can we play around with the quality settings to get a more decent image?

[umputun]

I still think this is not really due to scaling algo, but something about the ratio

[mullakhmetov]

I can't get any more decent scaling result even in Photoshop. I tried to crop source image to 1120x1120 and resize it to 140x140 (to prevent round off dimensions, which could cause text blurriness).

Do you have some references? Maybe it is impossible to do better for such a small destination image size?

[umputun]

this is how it looks on github. As you may see - perfectly readable

[umputun]

btw, if the problem related to the small final size (800/300 default) we can increase it in cli/env (ImageGroup.ResizeWidth and ResizeHeight)

[mullakhmetov]

As you may see - perfectly readable

Agree. But in your example image was scaled down with ~2.7 factor (from 1746x1790 to 663x647) while I use ~8 factor in comparison above.

Here's CatmullRom-scaled to the same as github's size:

Pretty close to the reference and certainly readable.

btw, if the problem related to the small final size

BiLinear algo is always much worse than CatmullRom, Lanczos3 is always a little better than the CatmullRom (in quality terms). But you are right, this is becoming a problem only with high scale factor.

(800/300 default)

Hmm. Image in original post downscaled to 143x300. Why so?

[paskal]

To clarify, GitHub image above is 1746x1790, it's scaled in render but is not altered in quality at all.

[mullakhmetov]

but is not altered in quality at all

Of course it did :) The scale factor is just too low to see quality degradation. Here's original image github-like scaled at browser-side to 400 width:

[umputun]

Hmm. Image in original post downscaled to 143x300. Why so?

https://github.com/umputun/remark/blob/master/backend/app/store/image/image.go#L174

[paskal]

but is not altered in quality at all

Of course it did :) The scale factor is just too low to see quality degradation. Here's original image github-like scaled at browser-side to 400 width:

I mean GitHub presents the original picture and it's up to the browser to scale it, and we are playing a completely different game here. If we do resize, let's not look at GitHub example and rather check what is desired quality vs size of a picture we are willing to store.

[umputun]

I'm not sure why browser scaling is a "completely different game". If it can downscale the image to the proper size and keep it readable, what prevents us to do a similar thing on the backend side?

[umputun]

just to clarify - I don't really care if we have to keep a larger image in order to make it readable. If the only way is to downscale it to some 1200/1200 - so be it

[mullakhmetov]

I still don't get why the OP image dimensions so small. They both are lower than the corresponding default limits. It is posible only if source image dimensions were under limits and resize make no sense. Or ImageGroup settings were different in the days when OP image was created.

[umputun]

as far as I recall dimensions adjusted on both X and Y. 300 is the max height so, it was downscaled from whateverW-whateverY to (max 800 W)- (max 300 Y)

[mullakhmetov]

Ohh, I am sorry, I misread default limits (at least twice) and thought they are 800x600.
Ok, 300px height limit for the portrait oriented screenshots is too small. 800x800 would be more reasonable.

[umputun]

setting 800x800 doesn't solve the problem by itself. The image is still blurry and hard to read. This is an example:

[mullakhmetov]

1. The interpolation algorithm definitely needs to be changed. Lanczos3 provides the best quality, but it's presented only in external dependencies: imaging for instance. If the new dependency for such thing is not acceptable, let's just switch to CatmullRom, otherwise let's chooseimaging.
2. Setting new limits doesn't affect anything above because the image in addition to server side scaling (with 800x800 limits) was scaled on browser side: https://github.com/umputun/remark/blob/master/frontend/app/components/raw-content/raw-content.scss#L28
   I don't know how to properly customize CSS and pass env vars there. Also I'm not sure raw-content.scss is the only place where IMAGE_RESIZE_* env vars should be handled.

[Ivajkin]

The problem is not scaling algorithms themselves but scaling to serve for retina displays.

[Ivajkin]

Please review and approve PR #526 (comprehensively tested).
@umputun

• Fixed "Improve quality of images Improve quality of images #488" (sharp vibrant awesome images)
• Added retina display support
• Support for 3x devices including 11 Pro and MacPro
• Resources usage is good as was for scaled images (change is subtle after scaling), example for big image is 180 KB and 298 KB.
• All checks have passed successfully ✅

Example

[paskal]

Can we resolve this one after merging #526?

[umputun]

not sure. Maybe switching to CatmullRom will give us something good. Will be nice to see side-by-side with high res resize

[mullakhmetov]

What do you mean by "high res resize"? Something like downscale 4000x3000 image with 2400x900 limits?

[mullakhmetov]

Source and ApproxBiLinear, BiLinear, CatmullRom and Lanczos3 for the reference from top to bottom side-by-side

I'm not sure why browser scaling is a "completely different game". If it can downscale the image to the proper size and keep it readable, what prevents us to do a similar thing on the backend side?

Browser knows device pixel ratio and use it to get physical linear resolution from logical height and width CSS properties. In other words <img src=http://example.com/img_900x900.png width="300" height="300"> may be scaled to different physical resolution on different displays.

[paskal]

Source and ApproxBiLinear, BiLinear, CatmullRom and Lanczos3 for the reference from top to bottom side-by-side

№1 is worst (too sharp), №3 and №4 are the same and better than №2 (too blurry) is you'd ask me.

[umputun]

What do you mean by "high res resize"? Something like downscale 4000x3000 image with 2400x900 limits?

Correct

[paskal]

@mullakhmetov, thank you very much! Would you be so kind as to repeat your research and show four algorithms side-by-side? If you don't mind, better as separate pictures so it would be possible to cycle through them.

[mullakhmetov]

It's not easy to find suitable (by license and for comparison) ~12mpx picture, as it turned out.

Original
ApproxBiLinear
BiLinear
CatmullRom
Lanczos3

[paskal]

• ApproxBiLinear still too sharp.
• BiLinear still too blurry in comparison with others.
• CatmullRom and Lanczos3 almost the same, and the size is almost the same. My vote is with sticking with one of them which is easier \ faster.

@umputun?

[mullakhmetov]

keep in mind there's no Lanczos3 implementation in stdlib

[paskal]

Let’s switch to CatmullRom then, could you please prepare a PR for that?

[mullakhmetov]

@paskal sure, see #582