Refer to this link for installing dependencies, librealsense2 and the build. https://dev.intelrealsense.com/docs/compiling-librealsense-for-linux-ubuntu-guide?_ga=2.42442656.40329647.1712875214-856519986.1712454558
- Everything is pretty much covered , check if the patching is done correctly by running this command:
sudo dmesg | tail -n 50
The log should indicate that a new uvcvideo driver has been registered.
Refer to Troubleshooting in case of errors/warning reports.
-
check the camera by running :
realsense-viewer -
For ros interface installation, follow these instructions from point NUMBER 4 https://zhaoxuhui.top/blog/2020/09/09/intel-realsense-d435i-installation-and-use.html
This part provides a detailed guide on setting up the ros_sam package in a ROS environment. Please follow the steps below to ensure a proper installation.
Ensure that you have a working ROS installation and that catkin is properly set up on your system. Additionally, you will need git and pip installed to handle repository cloning and Python packages installation.
Navigate to the src directory of your catkin workspace and clone the ros_sam repository using the following command:
cd ~/catkin_ws/src # Replace '~/catkin_ws' with your actual catkin workspace path
git clone https://github.com/robot-learning-freiburg/ros_sam.gitNavigate to the Segment-Anything Model Checkpoints page on GitHub. Download the required checkpoint files and place them in the ros_sam/model directory within your workspace.
Use pip to install the segment-anything library directly from its GitHub repository:
pip install git+https://github.com/facebookresearch/segment-anything.gitClone the rqt_image_view_seg repository into your catkin workspace's src directory:
git clone https://github.com/ipab-slmc/rqt_image_view_seg.git ~/catkin_ws/src/Clean your workspace and build the specific packages using catkin build:
cd ~/catkin_ws
catkin clean # Option
catkin build ros_sam rqt_image_view_segCopy the necessary script files from the ros_sam/scripts directory to the designated scripts folder in your workspace:
cp ~/catkin_ws/src/ros_sam/scripts/* ~/catkin_ws/src/ros_sam/scripts/Ensure to source your catkin workspace to apply all changes:
source ~/catkin_ws/devel/setup.bashOpen separate terminals and execute the following commands in each to run the system components:
- Start ROS core:
roscore- Launch the RealSense camera:
roslaunch realsense2_camera rs_camera.launch- Run the SAM nodes:
rosrun ros_sam sam_node.py
rosrun ros_sam node_subs.py # Alternatively, you can use sam_fix_p.py / sam_grid_p.py- Launch the GUI:
roslaunch ros_sam gui_test.launchYou can use rqt_plot to monitor various topics and nodes:
rosrun rqt_plot rqt_plotOn the GUI page, choose /camera/color/image_raw for input and /masked_images or /segmented/composite_image for the output to see the segmented images.
If you encounter errors during execution, here are some solutions:
- Missing Dependencies: Use rosdep to install any missing dependencies:
rosdep install --from-paths ~/catkin_ws/src --ignore-src --rosdistro=${ROS_DISTRO} -y-
GUI RuntimeError: If you get a ValueError:
PyCapsule_GetPointer called with incorrect name, check the following resource for a solution. -
Control_transfer returned error: If you are receiving the control_transfer returned warning only several times during launch and then they stop, or they occur one or two times a minute after launch, then this warning can be safely ignored. link
Navigate to the src directory of your catkin workspace and clone the ultralytics_ros repository using the following command:
cd ~/catkin_ws/src # Navigate to the source directory of your catkin workspace
GIT_LFS_SKIP_SMUDGE=1 git clone -b noetic-devel https://github.com/Alpaca-zip/ultralytics_ros.gitAfter cloning, install the required Python dependencies:
pip install -r ~/catkin_ws/src/ultralytics_ros/requirements.txtUse rosdep to install any additional dependencies:
cd ~/catkin_ws
rosdep install -r -y -i --from-paths .Build the cloned repository using catkin build:
catkin buildRefer to the Run section of the README in the ultralytics_ros repository. Set the input_topic parameter to /camera/color/image_raw when running the node. Commands will vary based on the specific launch files and nodes available in the ultralytics_ros package.
Example use:
roslaunch ultralytics_ros tracker.launch debug:=true input_topic:=/camera/color/image_raw yolo_model:=best.ptNavigate to the src directory of your catkin workspace and clone the aurco repository using the following command, remember to switch to noetic-devel before cloning:
git clone https://github.com/pal-robotics/aruco_ros.git
After cloning, navigate to the aruco_ros directory and git checkout 3.1.4.
Remember to catkin build and source devel/setup.bash at the end at the catkin_ws directory.
start realsense camera node
roslaunch realsense2_camera rs_camera.launch
Start the single.launch file
roslaunch aruco_ros single.launch
Use image_view to observe the recognition effect
rosrun image_view image_view image:=/aruco_single/result
Check the posture
rostopic echo /aruco_single/pose
The MarkerID is 582, and the Markersize is 0.034m, which can be changed in the single.launch file.
start realsense camera node
roslaunch realsense2_camera rs_camera.launch
launch the yolo
roslaunch ultralytics_ros tracker.launch debug:=true input_topic:=/camera/color/image_raw yolo_model:=epoch300.pt
Start the single.launch file
roslaunch aruco_ros single.launch
Use image_view to observe the recognition effect
rosrun image_view image_view image:=/aruco_single/result
rosrun the pipeline script
rosrun realsense2_camera pipeline.py
launch kinova arm
roslaunch kortex_driver kortex_driver.launch arm:=gen3_lite ip_address:=10.18.2.230
socks ID = 80 bottle ID = 39 blanket ID = 81