compare output and parameters

[jmspaggi]

ImageMagick version

6.9.13-12

Operating system, version and so on

Linux t14s 6.12.6-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.6-1 (2024-12-21) x86_64 GNU/Linux

Description

Hi,

I'm trying to locate one picture within another one. I know for sure the picture is present. I asked google for some help and I ended up with this:

$ time compare -metric RMSE /tmp/a.png /tmp/b.jpg -similarity-threshold 0 -dissimilarity-threshold 1 -subimage-search NULL:
366.7 (0.00559548) @ 65,124
real 1m9,851s
user 4m23,354s
sys 0m0,444s
$ time compare -metric RMSE /tmp/a2.png /tmp/b.jpg -similarity-threshold 0 -dissimilarity-threshold 1 -subimage-search NULL:
21051.2 (0.321221) @ 96,555

real 1m17,564s
user 4m59,518s
sys 0m0,448s

The first example is where one picture is a cropped of the other one saved as a JPG (So not binary identical) while the second are non-related pictures.

It works pretty well! I have two questions related to this. The first is related to the output.
What does 366.7 (0.00559548) @ 65,124 mean? I figured that 65,124 are the coordinates, but what are the other 2? Feels like the one in parenthesis is how similar are the two pictures. but why is it "just" 0.3 on the second one? Does is vary between 0 and 1?
The second question is related to the parameters. Given that I know for sure one picture is included on the other one, can I adjust the thresholds to make things faster?

I figured adjusting to those parameters:

$ time compare -metric RMSE /tmp/a.png /tmp/b.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
366.7 (0.00559548) @ 65,124
real 0m39,562s
user 2m36,478s
sys 0m0,140s

makes the things faster. But while I think I understand what similarity-threshold (Thanks to the documentation), dissimilarity-threshold is still a bit obscure to me. Is there a better value I should use for this parameter?

Thanks.

PS: Because I have 400k pictures to process, performance is important to me.

[snibgo]

Some documentation is at https://imagemagick.org/script/compare.php

For "-metric RMSE", the number in parentheses is the score of the best subimage match from 0.0 (exactly the same) to 1.0 (totally different). My rule of thumb is that an RMSE score of less than 0.01 means I can't easily see a difference between the images. Your score of 0.321221 for unrelated images means that the best subimage match was clearly a poor match.

The coordinates are the best location (lowest RMSE) of the top-left of the subimage within the large image.

Setting "-similarity-threshold 0.01" will stop the search early if any location has that score or better (ie lower). So this may not find the overall best match.

Setting "-dissimilarity-threshold 0.1" won't save any time. If the best result is worse than this score, IM will raise an error. The default threshold is less than 1.0, so IM can raise an error even when there is some (fairly poor) match, so I generally use "-dissimilarity-threshold 1".

If your IM includes FFTW, this may accelerate the RMSE search. (This is a very recent enhancement to IM.)

A multi-scale (aka pyramid) search can greatly improve performance. For example, if you resize both images to 50%, the search space reduces to 1/4, and the number of comparisons also reduces to 1/4, so the total effort reduces to 1/16. The cost is a lack of precision (the found location is plus or minus one), and a theoretical possibility of finding a false match. We can overcome the lack of precision by repeating the search at full size, but heavily cropping the large image so the search space is very small.

Even better, we can resize to 50% images to 50%, so the effort is 1/16 * 1/16 = 1/256. And we can keep going while the images are "reasonably" large. This massively accelerates searches.

Doing this work in a script is painful, so I have written Process module: multi-scale image search. But that needs you to build IM yourself.

[jmspaggi]

Ha! Excellent! Thanks.

How can I make sure my version of IM has FFTW? I can install a more recent package if needed.

Regarding the resizing, that's super smart! I will definitely try that and have a look at your script.

One last question, what does the first number in the output represent? 366.7 in my case. Also, can I just remove the dissimilarity-threshold parameter then?

Thanks,

JMS

[jmspaggi]

Here are the result with a 50% reduction:

$ /usr/bin/compare -metric RMSE /tmp/a.png /tmp/b.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-seartime ch NULL:
366.7 (0.00559548) @ 65,124
real 0m34,992s
user 2m19,542s
sys 0m0,204s

$ time /usr/bin/compare -metric RMSE /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
1211.3 (0.0184832) @ 33,62
real 0m3,033s
user 0m5,998s
sys 0m0,012s

Given the 50% ratio, second result will be 66,124, wich is just perfect! And the time moved from 2m19 to 5s!

[fmw42]

MSE, RMSE and PSNR are not yet accelerated by FFT processing. But we are currently working on them and they should be added in the very near term. Currently only NCC is accelerated by FFT processing.

The current beta has MSE and RMSE seemingly working and we are working on PSNR. But it is a beta. So it could change.

[jmspaggi]

But I need 7.x for that, correct? Because I tried on the version I have and it's 10 to 100 times slower :-/ (And I don't get the right output)

$ time /usr/bin/compare -metric NCC /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
compare: images too dissimilar `/tmp/a_small.png' @ error/compare.c/CompareImageCommand/1167.
real 1m12,464s
user 2m24,337s
sys 0m0,065s

[fmw42]

Yes, you need IM 7 to get the accelerated NCC that uses FFTW delegate. In particular 7.1.0-3 or higher. See ImageMagick/ImageMagick#3950

[jmspaggi]

Interestingly seems that I have 2 versions and both are "slow"

$ magick -version
Version: ImageMagick 7.1.1-43 Q16 x86_64 20241222 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP(4.5)
Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg jxl lcms lqr ltdl lzma openexr pangocairo png raw tiff webp wmf x xml zip zlib zstd
Compiler: gcc (14.2)

$ compare --version
Version: ImageMagick 6.9.13-12 Q16 x86_64 18420 https://legacy.imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP(4.5)
Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png raw tiff webp wmf x xml zlib

$ time magick compare -metric NCC /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
compare: images too dissimilar `/tmp/a_small.png' @ error/compare.c/CompareImagesCommand/1174.
real 3m29,849s
user 6m58,378s
sys 0m0,032s

[fmw42]

On IM 7 you run: magick compare, not compare

[jmspaggi]

I did that, but IM 7 is still slower:

$ time magick compare -metric RMSE /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
1049.01 (0.0160069) @ 33,62
real 0m7,608s
user 0m15,082s
sys 0m0,032s

$ time compare -metric RMSE /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
1211.3 (0.0184832) @ 33,62
real 0m3,133s
user 0m6,188s
sys 0m0,036s

7.x takes 7.6s and 6.x takes 3.1 seconds. Less than half. For the exact same command.

[snibgo]

How can I make sure my version of IM has FFTW?

Use "magick -version". For an installation on my laptop:

Version: ImageMagick 7.1.1-39 (Beta) Q32-HDRI x86_64 629938332:20240913 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenCL OpenMP(4.5)
Delegates (built-in): bzlib cairo fftw fontconfig freetype heic jbig jng jpeg lcms ltdl lzma pangocairo png raqm raw rsvg tiff webp wmf x xml zip zlib zstd
Compiler: gcc (11.3)

So my "Delegates" includes "fftw". I can't find the recent post about RMSE, but see fmw42's response.

... what does the first number in the output represent? 366.7 in my case.

This is the score on a scale of 0 to QuantumLevel, which is 65535 for Q16. Note that 366.7 / 65535 = 0.005595. As that number depends on the IM build, I would ignore it.

Also, can I just remove the dissimilarity-threshold parameter then?

Yes, but be prepared for IM raising an error. I generally use "-dissimilarity-threshold 1" because that takes no longer, and ensures I won't get that error.

And the time moved from 2m19 to 5s!

Good stuff.

$ time /usr/bin/compare -metric NCC /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
compare: images too dissimilar

This is an IM bug. NCC scores in reverse to RMSE: 0.0 is non-match, 1.0 is a match. But the dissimilar test is if the score is greater than the threshold, IM declares the images are too dissimilar. That's another reason to use "-dissimilarity-threshold 1".

[jmspaggi]

Ok cool! Thanks for all those details!

My 7.x version has fftw included. But it seems that my 6.x version is faster for this usecase:

$ time magick compare -metric RMSE /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
1049.01 (0.0160069) @ 33,62
real 0m7,567s
user 0m15,045s
sys 0m0,032s

$ time magick compare -metric NCC /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 1 -subimage-search NULL:
0.746804 @ 33,62
real 3m37,371s
user 7m13,982s
sys 0m0,060s

$ time compare -metric RMSE /tmp/a_small.png /tmp/b_small.jpg -similarity-threshold 0.01 -dissimilarity-threshold 0.1 -subimage-search NULL:
1211.3 (0.0184832) @ 33,62
real 0m3,092s
user 0m6,108s
sys 0m0,022s