Getting a stream of images from a camera to a device for processing is easier said then done, and with plethora of options available selecting the right pipeline is a separate challenge in itself.
For this particular project, after spending hours with the raspi camera module, I settled on using a IP camera to stream images directly to a Jetson TX series SOC, in this case a TX2.
After some web surfing, and looking at options, I settled on the Amcrest series.
Specifically the IP8M-2493EW, technically produced for use outdoors according to the description, but figured indoor use was equally possible.
Image Capture Setup and Pipeline
The setup would consist of streaming media from the IP camera using the real time streaming protocol, RTSP, a ROS node running on the TX2 subscribing to the URI and converting the stream to ROS image type, and publishing the ROS consumable image over a virtual private network where a subscriber sits on the server. (I plan to write on a separate note about how I got the images from the server running on a Digital Ocean droplet to the webapp using socket.io)
In order to accomplish this I needed to find the URI associated with the stream, which sounds trivial but difficult when documentation is sparse, and coding up a ROS package to consume the stream and publish results, with image processing done using openCV3.3.1.
RTSP stream URI
The URI needs to be in the form ‘rtsp://(username):(password)@(ip):(port)/cam/realmonitor?channel=1&subtype=0’ where the variables enclosed in () needs to be populated.
Conversion to ROS message
This URI is then used as the source for the cv::VideoCapture object. The
frames are read, resized, and converted to a ROS image of type
sensor_msgs::ImagePtr.
Find the full code here.
Summary
The ROS package is still incomplete and just the bare minimum and sufficient for this prototype project. The stream is relatively stable, and I have found no issues with the pipeline running on the TX2 for multiple days. Next steps for this particular component is to optimize for fps and consider security measures.