ORB-SLAM2 with Kinect

[marcelinomalmeidan]

Hey everyone, I am trying to use Kinect with the ORB-SLAM2.
I already went through the dataset examples, and they worked fine.
However, when trying to use with kinect, it doesn't work: the camera window displays "Waiting for Images".
I already checked rostopic list, which gives:
/camera/depth_registered/image_raw
/camera/rgb/image_raw
/rosout
/rosout_agg

The two first topics are the ones that should be used with ORB-SLAM2, right?

In addition, I already tried launching libfreenect:
roslaunch freenect_launch freenect.launch

I get a bunch more rostopics and I am even able to view rgb video stream from image_view:
rosrun image_view image_view image:=/camera/rgb/image_raw
However, I cannot view the depth image from /camera/depth_registered/image_raw, but I can view the one in the topic:
rosrun image_view image_view image:=/camera/depth/image

I tried to change the ros_rgbd.cc file
From: message_filters::Subscriber<sensor_msgs::Image> depth_sub(nh, "camera/depth_registered/image_raw", 1);
To: message_filters::Subscriber<sensor_msgs::Image> depth_sub(nh, "/camera/depth/image", 1);

However, I didn't it would work anyway (and it didn't). Neither of the above solved the problem.

When executing the rqt_graph, it seems like ORB_SLAM2 creates a node called RGBD that is getting the topic /camera/rgb/image_raw. The rqt_graph can be seen below:
https://www.dropbox.com/s/fri1ik0xtzw7a9e/rosgraph.png?dl=0
(seems like the link can only be accessed by copying and pasting into the browser)

Would any of you have a suggestion of what can I do

[marcelinomalmeidan]

Nobody has had any experience concerning OrbSlam2 + kinect? I still haven't figured it out =/

[poine]

i played with orbslam and kinect last week end. what is your question?

On Wed, Feb 10, 2016 at 12:44 AM, marcelinomalmeidan <
[email protected]> wrote:

Nobody has had any experience concerning OrbSlam2 + kinect? =/

—
Reply to this email directly or view it on GitHub
#18 (comment).

[marcelinomalmeidan]

@poine, I have the following questions:

• Did you use ORBSLAM 1 or 2?
• Did you install any package for kinect?
• What were the steps for you to launch OrbSLAM with kinect? roscore followed by OrbSLAM launch? Or did you run anything else such as the freenect package?
• Did you ever encounter the "Waiting for Images" displayed in the GUI that opens while running OrbSLAM2?

As I said in the initial post, I was able to run the OrbSLAM2 examples, but I wasn't able to run OrbSLAM2 in real time with my kinect camera.

[poine]

Hola

On Wed, Feb 10, 2016 at 2:24 AM, marcelinomalmeidan <
[email protected]> wrote:

@poine https://github.com/poine, I have the following questions:

• Did you use ORBSLAM 1 or 2?

2

• Did you install any package for kinect?

  http://wiki.ros.org/freenect_stack

  • What were the steps for you to launch OrbSLAM with kinect? roscore
    followed by OrbSLAM launch? Or did you run anything else such as the
    freenect package?

you need to start the freenect node, see the freenect_launch package

• Did you ever encounter the "Waiting for Images" displayed in the GUI
  that opens while running OrbSLAM2?

that happens if orb_slam is not receiving images on the topic it is
expecting them ( "/camera/rgb/image_raw" and
/camera/depth_registered/image_raw" )

this is my launch file:

<!-- use device registration -->
<arg name="depth_registration"              value="true" />
<arg name="rgb_processing"                  value="true" />
<arg name="ir_processing"                   value="false" />
<arg name="depth_processing"                value="false" />
<arg name="depth_registered_processing"     value="true" />
<arg name="disparity_processing"            value="false" />
<arg name="disparity_registered_processing" value="false" />
<arg name="sw_registered_processing"        value="false" />
<arg name="hw_registered_processing"        value="true" />

and this is my toy: https://goo.gl/photos/rWufHfHqFx1oJYdp8
https://goo.gl/photos/w5VtSNZr3oqqVyPk8

HtH

Poine

[marcelinomalmeidan]

Hi @poine! Thank you very much! It works now...

For future reference (if anyone in the future has trouble with the same thing), this is what I did:
Installed libfreenect:
sudo apt-get install libfreenect-dev
sudo apt-get install ros-indigo-freenect-launch

Then I created a file called kinect_orbslam.launch with the contents described by poine, and launched it through roslaunch kinect_orbslam.launch.

;-)

[poine]

@marcelinomalmeidan , you're very welcome. By the way, are you using a
kinect v1 or v2?

And if you have a minute, could you test that version of ORB_SLAM2 :
https://github.com/poine/ORB_SLAM2 ? It starts immediately. (the pull
request is here: #21)

On Fri, Feb 12, 2016 at 12:11 AM, marcelinomalmeidan <
[email protected]> wrote:

Hi @poine https://github.com/poine! Thank you very much! It works now...

For future reference (if anyone in the future has trouble with the same
thing), this is what I did:
Installed libfreenect:

—
Reply to this email directly or view it on GitHub
#18 (comment).

[marcelinomalmeidan]

Hi @poine, I am using kinect v1, why? Are you considering to use v2?
By the way, I just tried your modified version, it worked! The system launches much faster now, thank you!

Since you are also working with a similar setup, did you find out how to publish ORBSlam data into a ROS topic? Since I am new to ROS, I've been struggling a lot to figure out how to get pose of the system as an ouput of ORBSLAM.

[poine]

@marcelinomalmeidan

On Fri, Feb 12, 2016 at 5:48 PM, marcelinomalmeidan <
[email protected]> wrote:

Hi @poine https://github.com/poine, I am using kinect v1, why? Are you
considering to use v2?

I am looking at some Ebay auctions. Seems you can get v2 kinects for cheap.
So... yes, I might try that.

By the way, I just tried your modified version, it worked! The system
launches much faster now, thank you!

I am glad it worked for you, and you're very welcome.

Since you are also working with a similar setup, did you find out how to

publish ORBSlam data into a ROS topic? Since I am new to ROS, I've been
struggling a lot to figure out how to get pose of the system as an ouput of
ORBSLAM.

I am working on that at the moment. I did not push the code to github yet
as I changed a bunch of things to make the Pangolin viewer optional (I can
not run it on my robot) and reintroduce ROS publishers that were in the
first version of ORB SLAM. I would prefer to discuss with Raul and get the
startup patch accepted before submitting another (larger) patch.

I would like to formalize a generic viewer interface to ORB SLAM2 so that i
can use either Pangolin, or ROS, or nothing.
Let me know if you get desperate about ouptut to ROS and I'll push my
current code to github.

[marcelinomalmeidan]

@poine, I do have a Kinect v2 here (which, by the way, was never taken out of the box). I will try to use it later, I'll let you know the outcome!

I understand your concern about getting the startup patch accepted before submitting this new one. I am not desperate, but I would be curious to play with it. Take your time! ;-)

[sahabi]

I am working on that at the moment. I did not push the code to github yet
as I changed a bunch of things to make the Pangolin viewer optional (I can
not run it on my robot) and reintroduce ROS publishers that were in the
first version of ORB SLAM. I would prefer to discuss with Raul and get the
startup patch accepted before submitting another (larger) patch.

Hi @poine, is it possible to push that code to github? I'm really desperate to publish the output to ROS!

[poine]

@mohammed Alshiekh

ok, i will try to put something in my git. What is it exactly you want to
do? run without the pangolin graphical interface and publish ORBSLAM poses
to ROS?

On Fri, Mar 11, 2016 at 10:45 PM, Mohammed Alshiekh <
[email protected]> wrote:

I am working on that at the moment. I did not push the code to github yet
as I changed a bunch of things to make the Pangolin viewer optional (I can
not run it on my robot) and reintroduce ROS publishers that were in the
first version of ORB SLAM. I would prefer to discuss with Raul and get the
startup patch accepted before submitting another (larger) patch.

Hi @poine https://github.com/poine, is it possible to push that code to
github? I'm really desperate to publish the output to ROS!

—
Reply to this email directly or view it on GitHub
#18 (comment).

[sahabi]

@poine Yes, I'd like to have the poses published onto a ROS topic. Is it possible to have them both working at the same time? That is the graphical interface and ROS?

Thanks!

[fanyuzeng]

@poine @marcelinomalmeidan Great, thank you so much!

[Mrzzhao]

@poine @marcelinomalmeidan ,hi ,I am new to the orbslam2. I am trying to use kinect2 to run orbslam2.I followed the site :https://github.com/raulmur/ORB_SLAM2/blob/master/README.md. but it doesn`t work,the camera window displays "Waiting for Images". I already checked rostopic list, which gives:
/camera/depth_registered/image_raw
/camera/rgb/image_raw
/rosout
/rosout_agg.
I have been puzzled for several days.do you succeed in running orbslam2 with kinect2?Could you give me a favor? Thanks.

[chwimmer]

Hello @Mrzzhao,

it seems that you didn’t start the driver of the Kinect2. You only started ORB_SLAM2.
Thats easy to check if you type:
rostopic info /camera/rgb/image_raw
Now check if there is a Publisher. When I'm right there only should be a Subscriber.

[Mrzzhao]

@chwimmer ,thank you, I tried it ,the topic /camera/rgb/image_raw exists. And there are many types of messages. Today I tried another orbslam2 by Gaoxiang,and it works.Thank you again.Furthermore,I will try to solve this problem .I will tell you if I work it out.

[UThinkD]

@poine @marcelinomalmeidan @chwimmer Hi ,I meet the same problem! I am also trying to use Kinect2 to run ORBSLAM2.It works well when I choose TUM Dataset, but it doesn`t work when I use my Kinect2 ,the camera window displays "Waiting for Images".
As chwimmer said above ,before rosrun ,I have run "roslaunch kinect2_bridge kinect2_bridge.launch" ,but it doesn't work. If I still do not start driver of the Kinect2,than how should I do ? Please give me a favor? Thank you!

[chwimmer]

when you start

roslaunch kinect2_bridge kinect2_bridge.launch the topics here described are published. When you didn't do any remapping or change in the code ORBSLAM2 expects images at the topics:

/camera/rgb/image_raw and camera/depth_registered/image_raw

So the published topics and the subscribed topics dont correspond to each other. Thats why you got the
"Waiting for Images" state.

[UThinkD]

@chwimmer thakn you very much! It works now.
I change the topic from"/camera/rgb/image_raw and camera/depth_registered/image_raw" to the topics of kinect2, then ecompile.

[jahaniam]

You don't need to change it in the code and recompile, you can always convert one topic to another using ":=" while running ORBSLAM.
for example:
rosrun ORB_SLAM2 Mono Vocabulary/ORBvoc.txt Examples/Monocular/demo_cam.yaml /camera/image_raw:=/usb_cam/image_raw
This converts a published topic in /usb_cam/image_raw to the desired one , /camera/image_raw , in runtime.

[mantangL]

@UThinkD Hi, I'm meet the same problem!How did you change the topic from"/camera/rgb/image_raw and camera/depth_registered/image_raw" to the topics of kinect2? Please give me a favor?

[UThinkD]

mantangL <[email protected]>于2017年5月1日 周一17:23写道：

@UThinkD <https://github.com/UThinkD> Hi, I'm meet the same problem!How did you change the topic from"/camera/rgb/image_raw and camera/depth_registered/image_raw" to the topics of kinect2? Please give me a favor? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZBtGoWJi4nWFV1JPrazchhWHDOns1jXks5r1aSPgaJpZM4HUwfH> .

change the topic to topic of kinect2. You can type "topic list" to get the topics of kinect2. Which I choose is "/kinect2/sd/image_color_rect", 1) and /kinect2/sd/image_depth_rect", 1)"

[mantangL]

@UThinkD Thanks very much .I have solved this problem by your advice.And the introduce of ROS topic is :http://wiki.ros.org/cn/ROS/Tutorials/UnderstandingTopics

[tylerbuchman]

@poine did you make the patch? I also would like to see the published topics

[hesamira]

Hey everyone,
I am using Xtion PRO LIVE with the ORB-SLAM2. My ORB-SLAM2 does not work with the Xtion PRO LIVE. I used OpenNI2 to install Xtion PRO LIVE. My camera work with ROS, but it cannot work with ORB-SLAM2. I also tested my camera with libfreenect-dev , but it did not work with ORB-SLAM2.
I faced the following error:

[ INFO] [1514967208.178975139]: No devices connected.... waiting for devices to be connected

please guide me.