- Qt 6.12
- Qt Multimedia
- Audio Overview
- Qt 6.12.0 Reference Documentation

Multimedia Overview ◦ Video Overview

# Audio Overview

## Audio Features

Qt Multimedia offers a range of audio classes that cover both low and high level approaches to: audio input, output and processing.

## Audio Implementation Details

### Playing Compressed Audio

For playing media or audio files that are not simple, uncompressed audio, you can use the QMediaPlayer C++ class, or the MediaPlayer QML type. The QMediaPlayer class and associated QML types are also capable of playing video, if required.

See Supported Media Formats for more detail.

The media player needs to be connected to a QAudioOutput object (or the QML AudioOutput element) to play back audio.

Here is how you play a local file using C++:

     player = new QMediaPlayer;
     audioOutput = new QAudioOutput;
     player->setAudioOutput(audioOutput);
     // ...
     player->setSource(QUrl::fromLocalFile("/Users/me/Music/coolsong.mp3"));
     audioOutput->setVolume(0.5);
     player->play();

The same functionality in QML:

     MediaPlayer {
         audioOutput: AudioOutput {}
         source: "file:///path/to/my/music.mp3"
         Component.onCompleted: { play() }
     }

### Recording Audio to a File

To record audio to a file, you need to create a capture session and connect to it an audio input and a recorder. These elements are implemented with the QMediaCaptureSession, QAudioInput, and QMediaRecorder classes. The default constructed QAudioInput selects the system default audio input. The recorder controls the recording process with a simple record() and stop() functions. Additionally, you can use it to select the output location, audio encoder, or file container format.

A session recording audio from the default microphone would look as follows in C++:

     QMediaCaptureSession session;
     QAudioInput audioInput;
     session.setAudioInput(&audioInput);
     QMediaRecorder recorder;
     session.setRecorder(&recorder);
     recorder.setQuality(QMediaRecorder::HighQuality);
     recorder.setOutputLocation(QUrl::fromLocalFile("test.mp3"));
     recorder.record();

In QML, the same can be achieved by:

     CaptureSession {
         audioInput: AudioInput {}
         mediaRecorder: MediaRecorder {
             id: recorder
             outputLocation: "file:///path/to/test.mp3"
         }
         Component.onCompleted: { recorder.record() }
     }

QMediaCaptureSession also provides support for more complex use cases such as image capturing or video recording.

### Low Latency Sound Effects

In addition to raw access to sound devices, the QSoundEffect class (and SoundEffect QML type) offers a more abstract way to play sounds. This class allows you to specify a WAV format file, which can then be played with low latency when necessary.

You can adjust the:

- Number of loops in which a sound effect is played.
- Volume of the sound effect.
- Muting of the sound effect.

### Low Level Audio Input and Output

The C++ API of Qt Multimedia offers classes for raw access to audio input and output facilities, allowing applications to receive raw data from devices like microphones, and to write raw data to speakers or other devices. Generally these classes do not do any audio decoding, or other processing, but they can support different types of raw audio data.

The QAudioSink class offers raw audio data output, while QAudioSource offers raw audio data input. The available hardware determines what audio outputs and inputs are available.

#### Push and Pull using QIODevice

The low level audio classes can operate in two modes - `push` and `pull`. In `pull` mode, the audio device is started by giving it a QIODevice. For an output device, the QAudioSink class will pull data from the QIODevice (using QIODevice::read()) when more audio data is required. Conversely, for `pull` mode with QAudioSource, when audio data is available then the data will be written directly to the QIODevice.

In `push` mode, the audio device provides a QIODevice instance that can be written or read to as needed.

Note: The QIODevice does not immediately access the audio device, but instead buffers the data internally. This means that the QIODevice can be used to write or read data at any time from the application thread and buffering of typically 250ms is added. If the application is not delivering audio data to the QIODevice quickly enough (for QAudioSink) or is not reading the QIODevice quickly enough, audio dropouts will occur.

#### Callback-based Interface

In addition to using a QIODevice based interface, the low level audio classes provide a callback-based interface that allows users to register a callback which is called on the audio thread whenever the audio device needs or delivers more data. This allows much lower latency audio processing, as the application can process the data directly on the audio thread.

     {
         QAudioFormat format;
         // Set up the format, eg.
         format.setSampleRate(44100);
         format.setChannelCount(2);
         format.setSampleFormat(QAudioFormat::Float);

         QAudioDevice info(QMediaDevices::defaultAudioOutput());
         if (!info.isFormatSupported(format)) {
             qWarning() << "Raw audio format not supported by backend, cannot play audio.";
             return;
         }

         audio = new QAudioSink(format, this);
         float phaseIncrement = 2 * M_PI * 220.0 / format.sampleRate(); // 220 Hz sine wave
         audio->start([&phase, phaseIncrement] (QSpan<float> interleavedAudioBuffer) {
             // The audio callback should not call any functions that may potentially be blocking

             // Fill the audio buffer with a sine wave
             const int sampleCount = interleavedAudioBuffer.size() / 2; // Stereo, so divide by 2
             for (int i = 0; i < sampleCount; ++i) {
                 float sample = std::sin(phase);
                 interleavedAudioBuffer[i * 2] = sample;     // Left channel
                 interleavedAudioBuffer[i * 2 + 1] = sample; // Right channel
                 phase += phaseIncrement;                    // Increment phase for next sample
             }
         });

         if (!audio->error() == QtAudio::Error::NoError) {
             // in addition to the other start() signatures, starting the audio callback will fail if
             // * the backend does not implement callback-based IO (the API is available on all major
             //   platforms)
             // * the signature of the audio callback does not match format.sampleFormat()

             qWarning() << "Error starting audio output:" << audio->errorString();
         }
     }

Note: This API is only available on platforms that support the callback API: Apple's CoreAudio (macOS, iOS, etc), Windows, Linux (using the PulseAudio or PipeWire backend) and Android.

Note: The callback will be called on a soft-realtime audio thread. It is important to ensure that the callback does not block, as this can cause audio glitches or dropouts. This includes performing blocking IO, locking mutexes, allocating memories or any other operations that may block. For best practices consult Ross Bencina's article Real-time audio programming 101: time waits for nothing. Also consider using clang's Realtime sanitizer to validate the audio callback.

### Decoding Compressed Audio to Memory

In some cases you may want to decode a compressed audio file and do further processing yourself. For example, mixing multiple samples or using custom digital signal processing algorithms. QAudioDecoder supports decoding local files or data streams from QIODevice instances.

Here's an example of decoding a local file:

     QAudioFormat desiredFormat;
     desiredFormat.setChannelCount(2);
     desiredFormat.setSampleFormat(QAudioFormat::Int16);
     desiredFormat.setSampleRate(48000);

     QAudioDecoder *decoder = new QAudioDecoder(this);
     decoder->setAudioFormat(desiredFormat);
     decoder->setSource("level1.mp3");

     connect(decoder, &QAudioDecoder::bufferReady, this, &AudioDecodingExample::readBuffer);
     decoder->start();

     // Now wait for bufferReady() signal and call decoder->read()

### Spatial Audio

The Qt Spatial Audio module provides an API for implementation sound fields in 3D space.

## Reference Documentation

### C++ Classes

QAmbientSound | A stereo overlay sound
QAudioBuffer | Represents a collection of audio samples with a specific format and sample rate
QAudioBufferInput | Used for providing custom audio buffers to QMediaRecorder through QMediaCaptureSession
QAudioBufferOutput | Used for capturing audio data provided by QMediaPlayer
QAudioDecoder | Implements decoding audio
QAudioDevice | Information about audio devices and their functionality
QAudioEngine | Manages a three dimensional sound field
QAudioFormat | Stores audio stream parameter information
QAudioInput | Represents an input channel for audio
QAudioListener | Defines the position and orientation of the person listening to a sound field defined by QAudioEngine
QAudioOutput | Represents an output channel for audio
QAudioRoom
QAudioSink | Interface for sending audio data to an audio output device
QAudioSource | Interface for receiving audio data from an audio input device
QMediaCaptureSession | Allows capturing of audio and video content
QMediaRecorder | Used for encoding and recording a capture session
QSoundEffect | Way to play low latency sound effects
QSpatialSound | A sound object in 3D space
QtAudio | Contains enums used by the audio classes

### QML Types

AmbientSound | A stereo overlay sound
AudioEngine | Manages sound objects inside a 3D scene
AudioInput | An audio input to be used for capturing audio in a capture session
AudioListener | Defines the position and orientation of the person listening to a sound field defined by a AudioEngine
AudioOutput | An audio output to be used for playback or monitoring of a capture session
AudioRoom
CaptureSession | Allows capturing of audio and video content
MediaPlayer | Adds media playback to a scene
MediaRecorder | For encoding and recording media generated in a CaptureSession
PlaybackOptions | Low level media playback options
SoundEffect | Type provides a way to play sound effects in QML
SpatialSound | A sound object in 3D space
audioDevice | Describes an audio device
mediaMetaData | Provides meta-data for media files

### Examples

Audio Devices Example | List available audio devices and their configuration.
Audio Output Example | Enabling audio playback using the QAudioSink class.
Audio Recorder Example | Discovering the available devices and supported codecs.
Audio Source Example | Recording audio using the QAudioSource class.
Spatial Audio Panning Example | Shows some of the capabilities of the spatial audio engine in Qt

Multimedia Overview ◦ Video Overview