backend-notes-gstreamer.qdoc

Go to the documentation of this file.

// Copyright (C) 2024 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only
 
/*!
\page qtmultimedia-gstreamer.html
\title Qt Multimedia GStreamer backend
\brief Platform notes for the GStreamer backend
 
This page covers the limitations of the GStreamer backend of Qt Multimedia.
 
\section1 Limitations and Known Issues
 
GStreamer is not bundled with Qt, but it is typically deployed with the Linux distribution.
 
\list
\li Certain bugs may be due to the GStreamer version being used. We recommend to use the latest
  GStreamer bug fix release on your platform.
\li Certain bugs may also be related to the libraries used by GStreamer, like Pulseaudio. Most
 notably Pulseaudio v16 has a known bug that causes the GStreamer pipeline to hang and requires
  backports of these two patches:
  \list
    \li \l{https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/745}
    \li \l{https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/764}
  \endlist
  This bug currently affects most mainstream Linux distributions, including Ubuntu 22.04, 23.10 and
  24.04, Debian 11 and 12, as well as Fedora 39 and 40.
\li Seeking, playback rates, loop, switching sinks have known bugs.
\li Audio features requires PulseAudio. See \l{Qt Multimedia on Linux}{Linux platform notes}
  for details.
\endlist
 
\section1 Customization points
 
Qt Multimedia provides certain customization points to allow access to the underlying GStreamer
pipeline. These customization points are considered private APIs and may be subject to change. The
entry point is \c{class QGStreamerPlatformSpecificInterface}.
 
\section2 Raw pipeline access
 
The \c{GstPipeline} underlying the \l{QMediaPlayer} and \l{QMediaCaptureSession} can be accessed.
 
\warning This is an unsafe API, as the pipeline is still managed by the Qt implementation. Great
care is required when using this API.
 
\code
#include <QtMultimedia/private/qgstreamer_platformspecificinterface_p.h>
 
[...]
QMediaMediaPlayer player;
GstPipeline *pipeline = QGStreamerPlatformSpecificInterface::instance()->gstPipeline(&player);
[...]
QMediaCaptureSession session;
GstPipeline *pipeline = QGSreamerPlatformSpecificInterface::instance()->gstPipeline(&session);
\endcode
 
\section2 Custom GStreamer elements as sinks and sources
 
It is possible to create GStreamer elements from a GStreamer pipeline decription and wrap them
inside a \c{QCamera} or \c{QAudioDevice}:
 
\code
#include <QtMultimedia/private/qgstreamer_platformspecificinterface_p.h>
 
[...]
QByteArray pipelineString = "videotestsrc is-live=true ! gamma gamma=2.0";
 
QMediaCaptureSession session;
session.setVideoSink(wid.videoSink());
 
QCamera *cam = QGStreamerPlatformSpecificInterface::instance()->makeCustomGStreamerCamera(
         pipelineString, &session);
session.setCamera(cam);
\endcode
 
\section2 QMediaPlayer: custom sources
 
The \c{QMediaPlayer} accepts a GStreamer pipeline decription as source URI:
 
\code
QMediaPlayer player;
player.setSource(u"gstreamer-pipeline: videotestsrc name=testsrc"_s);
\endcode
 
This will try to compile the pipeline description to use as source in the QMediaPlayer and will be
automatically connected to the sinks of the QMediaPlayer.
 
\warning Hic sunt dracones! Custom pipelines are an experimental feature: the custom pipelines do
not map well to QMediaPlayer APIs, most notably the media status, metadata APIs, and transport state.
Most calls will directly map to the GStreamer pipeline, which can lead to undefined behavior
depending on the pipeline. In most cases, the \c{gstreamer-pipeline:} may not be the right choice
for application code: for arbitrary video sources, the \c{QMediaCaptureSession} with a custom camera
(see above) is the preferred choice. For arbitrarily complex pipelines that only want to draw into a
Qt/QML GUI, GStreamer's \c{qml6glsink} (see below) may be a more robust choice.
 
\section1 Architectural Considerations.
 
Qt Multimedia is not a general purpose streaming framework and not necessarily the architecturally
best way to use GStreamer with Qt. Developers, who need a high degree of control over the GStreamer
pipeline, but only want to show the video output Qt, may want to consider using GStreamer's
\l{https://gstreamer.freedesktop.org/documentation/qml6/index.html}{qml6glsink}.
 
 
 */