Devlog

Over the past few years, we've been slowly working on improving the platform-specific plugins for Windows, macOS, iOS, and Android, and making them work as well as the equivalent plugins on Linux. In this episode, we will look at audio device switching in the source and sink elements on macOS and Windows.

On Linux, if you're using the PulseAudio elements (both with the PulseAudio daemon and PipeWire), you get perfect device switching: quick, seamless, easy, and reliable. Simply set the device property whenever you want and you're off to the races. If the device gets unplugged, the pipeline will continue, and you will get notified of the unplug via the GST_MESSAGE_DEVICE_REMOVED bus message from GstDeviceMonitor so you can change the device.

As of a few weeks ago, the Windows Audio plugin wasapi2 implements the same behaviour. All you have to do is set the device property to whatever device you want (fetched using the GstDeviceMonitor API), at any time.

A merge request is open for adding the same feature to the macOS audio plugin, and is expected to be merged soon.

For graceful error handling, such as accidental device unplug or other unexpected errors, there's a new continue-on-error property. Setting that will cause the source to output silence after unplug, whereas the sink will simply discard the buffers. An element warning will be emitted to notify the app (alongside the GST_MESSAGE_DEVICE_REMOVED bus message if there was a hardware unplug), and the app can switch the device by setting the device property.

Thanks to Seungha and Piotr for working on this!

Thanks to the newly added atenc element, you can now use Apple's well-known AAC encoder directly in GStreamer!

gst-launch-1.0 -e audiotestsrc ! audio/x-raw,channels=2,rate=48000 ! atenc ! mp4mux ! filesink location=output.m4a

It supports all the usual rate control modes (CBR/LTA/VBR/CVBR), as well as settings relevant for each of them (target bitrate for CBR, perceived quality for VBR).

For now you can encode AAC-LC with up to 7.1 channels. Support for more AAC profiles and different output formats will be added in the future.

If you need decoding too, atdec is there to help and has supported AAC alongside a few other formats for a long time now.

GStreamer's VideoToolbox encoder recently gained support for encoding HEVC/H.265 videos containing an alpha channel.

A separate vtenc_h265a element has been added for this purpose. Assuming you're on macOS, you can use it like this:

gst-launch-1.0 -e videotestsrc ! alpha alpha=0.5 ! videoconvert ! vtenc_h265a ! mp4mux ! filesink location=alpha.mp4

Click here to see an example in action! It should work fine on macOS and iOS, in both Chrome and Safari. On other platforms it might not be displayed at all - compatibility is unfortunately still quite limited.

If your browser supports this format correctly, you will see a moving GStreamer logo on a constantly changing background - something like this. That background is entirely separate from the video and is generated using CSS.