Skip to main content Link Menu Expand (external link) Document Search Copy Copied
DL Docs

Insert Topic Here PyTorch

Created : 23/02/2023 | on Linux: 5.4.0-139-generic
Updated: 23/02/2023 | on Linux: 5.4.0-139-generic
Status: Draft

Using virtual video devices to redirect streams.

The aim of this exercise is to take a network stream (usually defined with an IP and a PORT with some information about the stream attributes) and place it in a virtual video device node (or a video loopback device) so it can be accessed similar to as a camera. e.g. share LVDS camera video via microsoft teams if you want to debug a camera over a video call.

Steps

  1. Create a virtual device.
  2. Link up the stream to the virtual device created in STEP 1.
  3. Test with a simple webcam application.
Prerequisites

Install v4l-utils

sudo apt install v4l-utils
Method 1

install v4l2loopback kernel modules (if you may get modprobe: FATAL: Module v4l2loopback not found in directory /lib/modules/x.x.x-xxx-generic you may want to try the second method)

sudo apt install v4l2loopback-dkms
Method 2

This time we install the module by compiling it. first get rid if any previous artifacts od v4l2loopback with sudo apt purge v4l2loopback-dkms

git clone git@github.com:umlaeute/v4l2loopback.git


Make sure you have met the dependencies

follow the build instructions

Note: run make clean if you have already run make before (in case of kernel change)

make && sudo make install
sudo depmod -a

Testing Installation

In the terminal

sudo modprobe v4l2loopback

If you want to assign a number and a name;

sudo modprobe v4l2loopback video_nr=10 card_label="virtual_lvds_camera"

exclusive caps (for input devices)

sudo modprobe v4l2loopback exclusive_caps=1

suggested command:

more info on the options can be found in the repository

sudo modprobe v4l2loopback video_nr=10 card_label="virtual_lvds_camera" exclusive_caps=1

With this method you will also get the v4l2loopback-ctl tool (compiled and added to your search path with the make install step). This will give you the ability to control FPS, image timeouts, placeholder picture, and setup caps (gstreamer caps)

1. Create a virtual device.

Create a device with device number:10 label: virtual_lvds_camera

sudo modprobe v4l2loopback video_nr=10 card_label="virtual_lvds_camera" exclusive_caps=1

2. Test the device

list devices

 v4l2-ctl --list-devices

check attributes

v4l2-ctl --device /dev/video0 --all

you may want to change the number (0 uses in the example) depending on the video_nr value used.

3. Creating and Accessing streams

Create a stream in a jetson device with STREAM_PORT and STREAM_IP set to the receiving device (i.e IP address and a port of the device we want to playback or create a virtual video device node)

gst-launch-1.0 v4l2src device=/dev/video0  !  nvvideoconvert  ! nvv4l2h264enc insert-sps-pps=true  iframeinterval=5 control-rate=1 maxperf-enable=true name=encoder ! h264parse config-interval=5 ! udpsink port=$STREAM_PORT host=$STREAM_IP sync=false

Then from the receiving device open a terminal and run the following to test the stream.

 gst-launch-1.0 udpsrc address=$STREAM_IP port=$STREAM_PORT ! h264parse ! avdec_h264 ! videoconvert ! autovideosink

At the time of writing there is a bug in v4l2sink that does not copy new buffers. one workaround is too add a tee please refer to this issue

gst-launch-1.0 videotestsrc ! tee name=t ! v4l2sink device=/dev/video0

However this is not an ideal fix as this causes the frame to freeze.

Next we can create a pipeline with a preview stream and a v4l2stream

setting caps for the v4l2 loopback device

v4l2loopback-ctl set-caps "video/x-raw, format=(string)YUY2, width=1280, height=720, fps=30/1" /dev/video0

gst-launch-1.0 udpsrc address=$STREAM_IP port=$STREAM_PORT ! h264parse ! avdec_h264 ! videoconvert ! tee name=t ! autovideosink   t. ! videoconvert  !   v4l2sink device=/dev/video0  async

Streaming and accessing remote devices.

sometimes you can be in a network configured by a network switch and you want to send a stream to a device that you just hooked up to the dumb switch on your desk. The device might have an ip that is not in the subnet of the network.

let’s say the devoce that you have introduced to the network has in ip 192.168.10.73 and you want to send a stream to that device. first your pc (or the node you dialed into (in the same network as the stream desination device)) needs to have an ip that can coounicate with it .

give it a statip ip with (here enp0s31f7 is you networ interface)

sudo ip addr add 192.168.10.45/24 dev enp0s31f7

now you can lau ch a pipeline from your device to the other device (the device not on your subnet)

This pipeline assume the device has a waylandsink and is accepating a h264 stream

gst-launch-1.0 videotestsrc ! tee name=t \
    t. ! queue ! videoconvert ! videoscale ! 'video/x-raw,width=320,height=240,format=I420' ! capsfilter caps="video/x-raw,width=320,height=240,format=I420" ! v4l2h264enc ! rtph264pay ! queue leaky=downstream ! udpsink port=5001 host=192.168.10.45 \
    t. ! queue ! waylandsink

a jpeg concoded stream

gst-launch-1.0 videotestsrc ! nvvideoconvert ! 'video/x-raw,format=I420,width=320,height=240,framerate=30/1' ! jpegenc ! rtpjpegpay ! udpsink port=5001 host=192.168.10.15

JPEG streaming using nvidia hardware accelerated nvvideoconvert element. (here we use a nvvideoconvert element with a capsfilter)

gst-launch-1.0 videotestsrc ! nvvideoconvert ! 'video/x-raw,format=I420,width=320,height=240,framerate=30/1' ! jpegenc ! rtpjpegpay ! udpsink port=5001 host=192.168.10.15

send from node tp pc

gst-launch-1.0 videotestsrc ! videoconvert !  videoscale ! 'video/x-raw,width=320,height=240,format=I420' !  capsfilter caps="video/x-raw,width=320,height=240,format=I420" ! v4l2h264enc !  rtph264pay ! udpsink port=5001 host=192.168.10.45

reciving a jpeg encoded stream

gst-launch-1.0 udpsrc port=5001 ! application/x-rtp, encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! videoconvert ! waylandsink

playing a udp strream

gst-launch-1.0 udpsrc  port=5001 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! rtph264depay ! v4l2h264dec ! videoconvert ! waylandsink


Check the next topic

Source: PyTorch Tutorial

Click here to report Errors, make Suggestions or Comments!