Benchmarking ORB_SLAM vs ORB_SLAM2

[MarcGyongyosi]

Hi @raulmur ,

I ran a few quick comparisons with your ROS example for ORB_SLAM2 in monocular mode and I can see a significant difference in performance between the previous version and this one. More specifically, the rate at which ORB_SLAM2 is able to process incoming frames on the same platform (now 3-4 vs previously 8-9 frames per second) is significantly lower. Also, ORB_SLAM2 refuses to initialize on the same dataset that ORB_SLAM is perfectly happy to initialize on.

What are your thoughts on this? What do you think are the major causes of this change?

Thanks a lot!

Marc

[raulmur]

Could you please provide a sequence where you notice a difference in performance so that I can analyse it? Provide also the settings for ORB-SLAM and ORB-SLAM2.
The reason for the initialization might be in the difference in the method to extract keypoints on the image. For some reason it might be less suitable for your sequence. In ORBextractor.cc line 1054 you can change to the old (ORB-SLAM) method to extract keypoints "ComputeKeyPointsOld()".
I do not see any reason for the lower frame rate. So please provide a sequence if possible.

[gitunit]

haven't tested for fps but i also had less good initializations with ORB_SLAM2 compared to ORB_SLAM. furthermore I've noticed that ORB_SLAM2 had more problems with almost pure rotations than ORB_SLAM. though changing to the old keypoints computation in line 1054 as you @raulmur suggested did help. i will provide a sequence for the significant parts soon but i can upload the full files too. the biggest is 17 GB (>10 min. with 30 fps in VGA). what would be the best free cloud place to upload such big files?

[cipri-tom]

@gitunit you could try Mega as they give 50 Gb for free.
[no advertising, just trying to be of help]

[gitunit]

@cipri-tom that's true, totally forgot about this one. thx. will upload soon.

[gitunit]

@raulmur
here [1] you'll find a 1 min. sequence where you can test the difference described above. it also includes the rotation where ORB_SLAM2 fails totally and ORB_SLAM1 only sometimes. i have also attached 2 setting files [2][3] with 2 different calibrations for the same camera (PSEYE) but there are no significant differences as far as i can tell, so use whatever you like. i hope that helps you to improve stuff.

[1] shortStreet.bag
[2] pseyeSettings_calibrated_with_openCV
[3] pseyeSettings_calibrated_with_KITTI_toolbox

[raulmur]

Thanks a lot @gitunit. This is really useful. I will take some time to look at this particular case and let you know my conclusions.