Extending GstHip for Inference and Custom GPU Processing on AMD and NVIDIA GPUs

In my previous post about GstHip, I introduced GStreamer's HIP backend for GPU processing on AMD and NVIDIA hardware. Since then, we have been extending the library to make it easier to integrate with inference workflows and to use in applications with custom GPU processing.

The improvements cover memory sharing, general-purpose GPU allocations, kernel execution, asynchronous resource management, and Rust bindings.

More Flexible Memory Sharing and Uploads

Moving data between a media pipeline and a GPU can introduce unnecessary copies. GstHip now supports importing DMA-BUF memory with a linear layout on AMD GPUs, allowing compatible buffers from other components to be accessed without first copying their contents through system memory. For example, compatible linear DMA-BUF output from GStreamer's VA video decoders can be imported as HIP device memory without copying the video frames. Imported resources are cached for reuse, and hipupload falls back to a CPU copy if importing is not possible.

Once imported, the frames can be processed with hipconvertscale for color conversion and scaling, or with custom HIP kernels for application-specific processing.

The hipupload element now allows upstream sources, such as a V4L2 camera source, to write directly into host-pinned memory optimized for GPU transfers. This can provide faster uploads than copying from regular system memory to HIP device memory.

From Video Frames to Inference Buffers

GstHip's memory APIs were originally designed around video frames. Inference workflows also need allocations for tensors and intermediate results, which do not necessarily have a video layout.

We have extended the allocation APIs to support arbitrary-sized linear buffers and separated video layout information from the public memory structure. This makes the same memory management infrastructure useful for general GPU computations as well as video processing. Support for floating-point and planar RGB formats also helps bridge video processing and model inputs.

The ONNX plugin now integrates HIP memory support, enabling zero-copy input sharing with a compatible GPU execution provider. Video preprocessing and inference can therefore keep data on the GPU, avoiding a round trip through system memory. Depending on the input layout and model requirements, GPU-side packing or normalization may still be needed.

Easier Integration of Custom GPU Kernels

C/C++ applications could already run custom GPU kernels using native HIP APIs. GstHip now provides a higher-level abstraction through GstHipKernelModule, covering runtime compiler loading, source compilation, compiled module loading, entry point lookup, and kernel launches on a GstHipStream.

This layer works with GStreamer's HIP memory and synchronization APIs and can be combined with the asynchronous resource cleanup described below.

The accompanying GstHipKernelArgs API handles kernel arguments, including GStreamer memory objects and scalar values. Together, these APIs reduce the backend-specific setup needed in applications and plugins. They also make kernel execution easier to expose through language bindings, allowing Rust applications to use GStreamer memory and typed scalar arguments without manually constructing the native kernel argument array.

The existing video converter now uses this layer too, and GstHipConverter itself has moved into the public library so applications can reuse its format conversion and scaling functionality.

Keeping Resource Cleanup off the Streaming Thread

Submitting GPU work asynchronously is only part of keeping a media pipeline moving. Resources must remain alive until the GPU has finished using them, and waiting for completion just to release those resources can stall the streaming thread.

GstHip now provides event completion notifications backed by internal worker threads. Applications can associate resources with a completion event and have them released in the background once the GPU work finishes. GstHipFenceData provides a convenient container for collecting those resources.

This moves completion waits and deferred resource cleanup away from the thread processing media, helping preserve its throughput. Dependencies between GPU streams can also be handled with GPU-side event waits, reducing the need for CPU synchronization between processing stages.

HIP Support from Rust

The new gstreamer-hip crate brings GstHip to Rust, including memory management, streams, events, kernel execution, and video conversion.

This allows Rust applications and GStreamer plugins to use the same GPU infrastructure without implementing their own backend dispatch and resource management. The bindings combine generated APIs with manual wrappers for operations that need additional ownership and lifetime handling.

These utilities can also be used outside a complete media pipeline: applications can use GstHip to manage GPU buffers and run their own processing kernels.

The following excerpt illustrates kernel compilation and launch, with initialization and memory setup omitted. The kernel entry point is named transform:

let module = gst_hip::HipKernelModule::from_source(
    &device,
    /* Write your kernel code here */,
    &[],
)
.unwrap();

// ... allocate and prepare input/output HIP memory ...

// Build the kernel arguments and launch the kernel.
let mut args = gst_hip::HipKernelArgs::new();
let launch_result = unsafe {
    args.add_memory(&input, 0).unwrap();
    args.add_memory(&output, 0).unwrap();
    args.add(COUNT);
    module.launch(
        "transform",
        Some(&stream),
        [COUNT.div_ceil(BLOCK_SIZE), 1, 1],
        [BLOCK_SIZE, 1, 1],
        &args,
    )
};
assert!(stream.synchronize());
launch_result.unwrap();

See the full Rust example for kernel compilation, memory allocation, upload, and reading back the results.

Give It a Try

These improvements are now available in the GStreamer main branch and will be included in the upcoming GStreamer 1.30 stable release series. If you're building GPU processing workflows with GStreamer, give it a try and let us know how it works for your use case.

If you have questions or need help with your project, please feel free to get in touch.