New udpsrc2 element
Over the past few years, I have worked on a new GStreamer UDP source element. This is
finally merged
now and will be part of both the GStreamer 1.30.0 release and the gst-plugins-rs 0.16.0 release.
The old element uses GIO for networking, which is quite inefficient by design.
The new implementation uses about 50% less CPU on my machine compared to the
old element for a 3 Gbit/s stream.
As can be seen from the docs
of the new element, it preserves the API of the old element. As such it should
generally be possible to use it as a drop-in replacement.
In addition to performance improvements, the new element also includes various
other improvements:
-
Support for faster packet receiving via Generic Receive Offload (GRO) on
Linux, and for using recvmmsg() on platforms where it is available to
significantly improve receive performance.
-
Complete support for multicast source filtering, including negative filters,
and support for platforms that do not have APIs for the IGMPv3 SSM
mechanism.
-
Always obtaining kernel-side packet receive times if available, which was
opt-in in the old element due to GIO performance issues with socket
control messages.
-
New preserve-packetization property that allows outputting multiple
packets in the same buffer, which improves performance for formats like
MPEG-TS where the UDP packetization is not necessary.
Give it a try with your pipelines and workloads and share your feedback or any
issues you encounter.
In the future, io_uring support on Linux could be added for even better
receive performance.
SMPTE ST2110 capture
While udpsrc2 is an improvement in general, its primary motivation is better
SMPTE ST2110 support in GStreamer. The old element could not handle the packet
rates typically used for such streams very well.
ST2110 defines a UDP/RTP-based set of
standards for transmitting raw or very-high bitrate audio / video / ancillary data
over Ethernet. It is intended as a replacement for SDI.
Related to this, we recently also merged some other improvements:
For all the new depayloaders there are also new, improved implementations of the
corresponding payloaders available.
Together, these improvements enable reliable ST2110 stream capture in GStreamer.
An example pipeline putting it all together would look as follows:
$ gst-launch-1.0 \
\
udpsrc2 address=239.255.64.20 port=16388 multicast-iface=enp15s0 buffer-size=20000000 caps='application/x-rtp, media=video, payload=96, clock-rate=90000, encoding-name=RAW, sampling=YCbCr-4:2:2, depth=10, width=1920, height=1080, exactframerate=60, colorimetry=BT709, pm=2110GPM, ssn=ST2110-20:2017, tp=2110TPN, a-sendonly="", a-ts-refclk="ptp=IEEE1588-2008:7C-2E-0D-FF-FE-1C-81-14:127", a-mediaclk="direct=0", ssrc-327995485-cname=E055FF0F3D6E4B349F7B786D8B6C837B' ! \
rtprecv latency=0 ! queue max-size-bytes=0 max-size-buffers=0 max-size-time=500000000 ! rtpvrawdepay2 ! \
\
\
udpsrc2 address=239.255.64.20 port=16386 multicast-iface=enp15s0 buffer-size=20000000 caps='application/x-rtp, media=video, payload=98, clock-rate=90000, encoding-name=SMPTE291, vpid_code=138, a-sendonly="", a-ts-refclk="ptp=IEEE1588-2008:7C-2E-0D-FF-FE-1C-81-14:127", a-mediaclk="direct=0", ssrc-2672978631-cname=E055FF0F3D6E4B349F7B786D8B6C837B' ! \
rtprecv latency=0 ! queue max-size-bytes=0 max-size-buffers=0 max-size-time=500000000 ! rtpsmpte291depay ! combiner.st2038 \
\
\
st2038combiner name=combiner start-time-selection=first ! videoconvert ! queue max-size-bytes=0 max-size-time=0 max-size-buffers=3 ! autovideosink \
\
\
udpsrc2 address=239.255.64.20 port=16384 multicast-iface=enp15s0 buffer-size=20000000 caps='application/x-rtp, media=audio, payload=(int)97, clock-rate=48000, encoding-name=(string)L24, encoding-params=64, a-sendonly="", a-ptime=0.125, a-ts-refclk="ptp\=IEEE1588-2008:7C-2E-0D-FF-FE-1C-81-14:127", a-mediaclk="direct\=0", ssrc-603238248-cname=(string)E055FF0F3D6E4B349F7B786D8B6C837B' \
rtprecv latency=0 ! queue max-size-bytes=0 max-size-buffers=0 max-size-time=500000000 ! rtpL24depay2 ! audioconvert ! autoaudiosink
This pipeline receives a 1080p60 4:2:2 YUV 10-bit video stream, ST291
ancillary data, and a 24-bit 48kHz 64-channel PCM audio stream. The video and
ancillary data are combined to a single stream, and then both the combined
video-ancillary stream and the audio are output.
rtprecv is used here for translating packet capture timestamps and RTP
header timestamps to consistent GStreamer timestamps.
Ancillary data
The pipeline above captures all three streams and merges the ancillary data
stream with the video. The ancillary data itself is not processed further.
One way to process the ancillary data further is to extract ST12 timecodes
from it and overlay them over the video.
For this, insert the following elements before the video sink:
... ! timecodestamper source=ancillary-meta ancillary-meta-locations='8:2000,570:2000' \
! videoconvert ! timeoverlay time-mode=time-code \
! autovideosink
Here timecodes from ancillary data at positions (8,2000) and (570,2000)
would be extracted and converted to GstVideoTimeCodeMeta on the video
buffers.
We recently added support for extracting ST12 timecodes from ancillary
meta as well.
The positions depend on the video signal standard in use and can be found in
the ST12 specifications.
