[keyframeselection] Algorithms are rewritten #1213

servantftechnicolor · 2022-09-05T09:43:34Z

Keyframe selection was not working previously.

This PR tries a new set of algorithms to replace the previous one

A simple one which simply perform a regular sampling on the temporal dimension
A more complex one which take the frame with the sharper value in a window, and check that this frame has sufficient motion with the previous selection frame.

fabiencastan · 2022-09-05T18:27:59Z

#include <opencv2/optflow.h> is not found in the linux CI.
Should we add a module here? https://github.com/alicevision/AliceVision/blob/develop/CMakeLists.txt#L687

fabiencastan · 2022-09-16T22:41:47Z

Do we really need highgui here?
keyframe/KeyframeSelector.cpp:20:31: fatal error: opencv2/highgui.hpp: No such file or directory

src/aliceVision/keyframe/KeyframeSelector.cpp

mh0g · 2022-09-30T07:52:50Z

simple mode tested and works

p12tic · 2022-10-03T02:43:10Z

src/aliceVision/keyframe/KeyframeSelector.cpp

-  std::mt19937 randomTwEngine(rd()); // standard mersenne_twister_engine seeded with rd()
-  std::uniform_int_distribution<> randomDist(0, std::numeric_limits<int>::max());
-  return randomDist(randomTwEngine);
+    std::random_device rd;             // will be used to obtain a seed for the random number engine


This will make testing unreliable. It's best to supply the source of randomness as a parameter so that in tests fake implementation can be used.

src/aliceVision/keyframe/KeyframeSelector.cpp

cbentejac

In simple mode, if maxNbOutFrame is set, the number of extracted frames is always equal to maxNbOutFrame+1.

The "smart" mode seems to be working correctly!

src/aliceVision/keyframe/KeyframeSelector.hpp

mh0g · 2022-10-25T11:25:36Z

Tested simple and advanced mode on windows, seems to work as intended.
Note: cmake option ALICEVISION_USE_OPENCV is off by default, to be working off the shelf, cmake needs to be called with -DALICEVISION_USE_OPENCV=ON

mh0g

Tested on windows, seems to work as intended

fabiencastan · 2022-12-07T10:51:24Z

src/aliceVision/keyframe/KeyframeSelector.cpp

-    {
-      ALICEVISION_LOG_ERROR("Cannot initialize the FeedProvider with " << path);
-      throw std::invalid_argument("Cannot while initialize the FeedProvider with " + path);
+            maxstd = std::max(maxstd, std_2);


If I understand correctly, with the max here, one pixel will give the score for the whole image? If that's the case, this does not sound reliable.

…er but works. Still to be tested carefully.

Address PR reviews: - give more explicit names to some variables - harmonize coding style - avoid some useless computations

- If _maxOutFrame is not 0 and _minFrameStep is superior to 1, stop ignoring _minFrameStep, take it into account and always enforce it - Ensure that there will be no more than _maxOutFrame selected frames when _maxOutFrame is not 0

Use the same indentation (4 spaces) for both the KeyframeSelector header and source files.

cbentejac · 2023-01-25T17:36:36Z

Replaced by #1343.

servantftechnicolor requested review from fabiencastan and fcastan September 5, 2022 09:43

servantftechnicolor mentioned this pull request Sep 5, 2022

[node] keyframeselection update with simpler algorithms alicevision/Meshroom#1765

Closed

fabiencastan added this to the 2.5.0 milestone Sep 5, 2022

fabiencastan mentioned this pull request Sep 15, 2022

[CI] Upgrade deps: opencv, expat #1224

Merged

fabiencastan force-pushed the dev/keyframeSelection branch from e54f2a4 to 85db23e Compare September 16, 2022 21:34

fabiencastan reviewed Sep 16, 2022

View reviewed changes

src/aliceVision/keyframe/KeyframeSelector.cpp Show resolved Hide resolved

servantftechnicolor force-pushed the dev/keyframeSelection branch from 85db23e to e70ca45 Compare September 26, 2022 12:00

fabiencastan requested a review from mh0g September 26, 2022 16:32

servantftechnicolor force-pushed the dev/keyframeSelection branch from e70ca45 to a5d4d57 Compare September 27, 2022 16:05

p12tic reviewed Oct 3, 2022

View reviewed changes

servantftechnicolor force-pushed the dev/keyframeSelection branch from 3105cab to 8665463 Compare October 7, 2022 13:09

p12tic reviewed Oct 8, 2022

View reviewed changes

src/aliceVision/keyframe/KeyframeSelector.cpp Outdated Show resolved Hide resolved

p12tic reviewed Oct 8, 2022

View reviewed changes

src/aliceVision/keyframe/KeyframeSelector.cpp Outdated Show resolved Hide resolved

p12tic reviewed Oct 8, 2022

View reviewed changes

src/aliceVision/keyframe/KeyframeSelector.cpp Outdated Show resolved Hide resolved

p12tic reviewed Oct 8, 2022

View reviewed changes

src/aliceVision/keyframe/KeyframeSelector.cpp Outdated Show resolved Hide resolved

cbentejac reviewed Oct 10, 2022

View reviewed changes

src/aliceVision/keyframe/KeyframeSelector.hpp Outdated Show resolved Hide resolved

mh0g previously approved these changes Oct 25, 2022

View reviewed changes

cbentejac dismissed mh0g’s stale review via 32f216e November 17, 2022 17:04

cbentejac force-pushed the dev/keyframeSelection branch 3 times, most recently from 724a040 to 3a988db Compare November 23, 2022 10:09

cbentejac force-pushed the dev/keyframeSelection branch from 8969a23 to 7b2f0d1 Compare December 6, 2022 17:31

fabiencastan reviewed Dec 7, 2022

View reviewed changes

servantftechnicolor added 2 commits December 19, 2022 10:32

[keyframeselection] Algorithms are rewritten. New logic is much simpl…

48d42ef

…er but works. Still to be tested carefully.

make sure we move correctly in the sequence after selecting a frame

d62b6ba

servantftechnicolor and others added 10 commits December 19, 2022 10:32

[keyframe] Remove some errors in constants used

025e4f9

[cmake] update cmake to use opencv for keyframe selection

89f64fd

[keyframe] make sure max == 0 works as intented

e3317d4

[videofeed] better error handling in goto frame

484c907

[keyframe] better error handling

a5bbabf

[keyframe] Fix typos in KeyframeSelector's docstrings

4df6329

[keyframe] Rename variables more explicitly

586c5d2

Address PR reviews: - give more explicit names to some variables - harmonize coding style - avoid some useless computations

[keyframe] Rename "simple" method to "regular"

ca70afd

[keyframe] Update the regular algorithm

90af6c9

- If _maxOutFrame is not 0 and _minFrameStep is superior to 1, stop ignoring _minFrameStep, take it into account and always enforce it - Ensure that there will be no more than _maxOutFrame selected frames when _maxOutFrame is not 0

[keyframe] Align the header's indentation with the source's

fdbd567

Use the same indentation (4 spaces) for both the KeyframeSelector header and source files.

cbentejac force-pushed the dev/keyframeSelection branch from 90e18ab to fdbd567 Compare December 19, 2022 12:52

fabiencastan marked this pull request as draft January 2, 2023 18:52

cbentejac mentioned this pull request Jan 25, 2023

Keyframe Selection: Rework and add new selection methods #1343

Merged

5 tasks

cbentejac closed this Jan 25, 2023

cbentejac deleted the dev/keyframeSelection branch August 22, 2023 16:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[keyframeselection] Algorithms are rewritten #1213

[keyframeselection] Algorithms are rewritten #1213

servantftechnicolor commented Sep 5, 2022

fabiencastan commented Sep 5, 2022 •

edited

Loading

fabiencastan commented Sep 16, 2022

mh0g commented Sep 30, 2022

p12tic Oct 3, 2022

cbentejac left a comment

mh0g commented Oct 25, 2022

mh0g left a comment •

edited

Loading

fabiencastan Dec 7, 2022

cbentejac commented Jan 25, 2023

[keyframeselection] Algorithms are rewritten #1213

[keyframeselection] Algorithms are rewritten #1213

Conversation

servantftechnicolor commented Sep 5, 2022

fabiencastan commented Sep 5, 2022 • edited Loading

fabiencastan commented Sep 16, 2022

mh0g commented Sep 30, 2022

p12tic Oct 3, 2022

Choose a reason for hiding this comment

cbentejac left a comment

Choose a reason for hiding this comment

mh0g commented Oct 25, 2022

mh0g left a comment • edited Loading

Choose a reason for hiding this comment

fabiencastan Dec 7, 2022

Choose a reason for hiding this comment

cbentejac commented Jan 25, 2023

fabiencastan commented Sep 5, 2022 •

edited

Loading

mh0g left a comment •

edited

Loading