WIP
Notes on working with D435 realsense, OpenVino, ROS 2.
| Camera Model: | D435 |
| librealsense Version: | v2.16.5 |
| Firmware Versions: | 05.12.06.00 |
| OS & Version: | Ubuntu 18.04.4 LTS |
| Kernel Version: | 4.15.0-112-generic |
ROS Distro: Dashing note: lsb_release outputs LTS but MATE was installed on the NUC.
Goal: To publish 2d & 3d images from a D435 to a ros topic that would be consumed by an OpenVino machine vision pipeline with intentions to use with a Turtlebot2.
Status/Results: While I have been successful in working with 2d images, 3d is yet to work. [3/Aug/2020]
$nuc@nuc: ros2 run realsense_ros2_camera realsense_ros2_camera
$remote_pc@remote_pc: ros2 launch pipeline_object_topic.launch.py
Remember to update the yaml file in /params that the launch file loads with a path to the model, and associated labels.
FPS: The fps is very unstable having a range of 4-15fps for 2D object detection when using the D435. The fps is unstable especially when compared with the IpCamera input where the system was much more stable.
Getting the Intel Realsense camera D435 to work has been more involved then I thought, and a review of issues at the official repository suggests difficulties in integrating the device has not been as straightforward for others as well.
Working toward a solution involved the following steps:
- Downgrading kernel version to 4.15* after trying 5.3* unsuccessfully.
- Update firmware.
- Run patches and install dependencies.
- Moving everything to work with Dashing and installing ros2 packages related to realsense.
- After device side was resolved to an acceptable point, remapping the camera topic to appropriate
INPUT_TOPICdefined in the source was required.
Follow up items
- Calibrate IMU: https://github.com/IntelRealSense/librealsense/issues/5166
- Try with RealSenseCamera input in place of RealSenseCameraTopic by running OpenVino pipeline on the NUC.
- Debug below warnings when running realsense-viewer with 3D.
03/08 15:01:39,022 WARNING [139875290420992] (sensor.cpp:338) Unregistered Media formats : [ UYVY ]; Supported: [ ]
03/08 15:02:04,803 WARNING [139875103786752] (backend-v4l2.cpp:1013) Frames didn't arrived within 5 seconds
03/08 15:02:09,809 WARNING [139875103786752] (backend-v4l2.cpp:1013) Frames didn't arrived within 5 seconds
03/08 15:02:14,814 WARNING [139875103786752] (backend-v4l2.cpp:1013) Frames didn't arrived within 5 seconds
Command line reminder:
lsusb -v
usb-devices
uname -r
lsb_release -a
dmesg
rs-fw-update -l
v4l2-ctl --list-device
realsense-viewer
ros2 topic list -t
References
https://kowalczyk.me/change-default-kernel-in-ubuntu-18-04/ https://github.com/IntelRealSense/librealsense/blob/master/doc/distribution_linux.md https://github.com/IntelRealSense/librealsense/issues/1225 https://github.com/IntelRealSense/librealsense/issues/5598 https://github.com/intel/ros2_intel_realsense https://dev.intelrealsense.com/docs/firmware-update-tool https://qiita.com/t_kumazawa/items/eb3a60f0ca1fbca70bee https://naonaorange.hatenablog.com/entry/2018/11/04/174745