mediaengine

A Rust Swiss army knife for media. mediaengine reads media containers, decodes audio, and reads tags in safe, pure Rust, and grows into the other tools a media application needs, video included. It is the media engine of the PlayControl player.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

What it does today

• Demuxing: AIFF, CAF, ISO/MP4 (MP4, M4A, MOV), MKV/WebM, Ogg and Wave, with automatic format detection.
• Audio decoding: AAC-LC, ADPCM, ALAC, FLAC, MP1, MP2, MP3, PCM and Vorbis.
• Gapless playback: encoder delay and padding are trimmed by default, including AAC in MP4 through the edit list.
• Tags: APEv1, APEv2, ID3v1, ID3v2, MP4, RIFF and Vorbis comments.
• Video: the MP4 and MKV demuxers expose video tracks with their codec parameters and packets (for example H.264, HEVC, AV1, VP8 and VP9). Video decoding is not implemented yet.
• Safe Rust: the library crates contain no unsafe code.

Containers

Format	Gapless	Feature flag	Default	Crate
AIFF	Yes	aiff	No	mediaengine-format-riff
CAF	No	caf	No	mediaengine-format-caf
ISO/MP4	Yes	isomp4	No	mediaengine-format-isomp4
MKV/WebM	No	mkv	Yes	mediaengine-format-mkv
Ogg	Yes	ogg	Yes	mediaengine-format-ogg
Wave	Yes	wav	Yes	mediaengine-format-riff

Gapless playback needs support from both the demuxer and the decoder. all-formats enables every container.

Audio codecs

Codec	Gapless	Feature flag	Default	Crate
AAC-LC	Yes	aac	No	mediaengine-codec-aac
ADPCM	Yes	adpcm	Yes	mediaengine-codec-adpcm
ALAC	Yes	alac	No	mediaengine-codec-alac
FLAC	Yes	flac	Yes	mediaengine-bundle-flac
MP1	No	mp1, mpa	No	mediaengine-bundle-mp3
MP2	No	mp2, mpa	No	mediaengine-bundle-mp3
MP3	Yes	mp3, mpa	No	mediaengine-bundle-mp3
Opus	Yes	opus	Yes	mediaengine-codec-opus
PCM	Yes	pcm	Yes	mediaengine-codec-pcm
Vorbis	Yes	vorbis	Yes	mediaengine-codec-vorbis

all-codecs enables every codec, mpa enables MP1, MP2 and MP3. HE-AAC and WavPack are not implemented. The Opus decoder decodes SILK-only, hybrid and CELT-only frames with their mode transitions, channel mapping families 0 and 1 (multistream), output gain and trims, and meets the RFC 8251 conformance tests. It conceals frames of 0 or 1 bytes (DTX and the empty packets that fill container gaps) and switches to and from CELT-only without redundancy by the reference decoder's method: SILK concealment matches it bit for bit, CELT concealment matches its level within 1 dB in every 2.5 ms block (its samples differ by floating-point rounding, which the periodic extrapolation amplifies). A lost packet cannot be signalled to the decoder: a gap reaches it only as such empty frames.

Tags

All tag readers live in mediaengine-metadata. Tags that are part of a container (MP4, RIFF, Vorbis comments in FLAC and Ogg) are read by that container's demuxer. The standalone formats have feature flags, all enabled by default: ape, id3v1 and id3v2 (together all-meta).

Other feature flags

• all: every container, codec and tag format.
• opt-simd (default): SIMD optimisations, split into opt-simd-sse, opt-simd-avx and opt-simd-neon. They pull in rustfft.
• exp-video-codecs and exp-subtitle-codecs: experimental decoder traits for video and subtitles.

Install

mediaengine is not published to a registry yet. Its crates are versioned 0.1.0. The facade crate mediaengine re-exports the core types as mediaengine::core, registers the enabled demuxers and decoders in mediaengine::default, and enables the formats and codecs through the feature flags above.

Usage

Decode the default audio track of a file:

use std::fs::File;

use mediaengine::core::codecs::audio::AudioDecoderOptions;
use mediaengine::core::formats::probe::Hint;
use mediaengine::core::formats::{FormatOptions, TrackType};
use mediaengine::core::io::MediaSourceStream;
use mediaengine::core::meta::MetadataOptions;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let path = std::env::args().nth(1).ok_or("usage: decode <file>")?;
    let mss = MediaSourceStream::new(Box::new(File::open(path)?), Default::default());

    // Detect the container format and open a reader for it.
    let mut format = mediaengine::default::get_probe().probe(
        &Hint::new(),
        mss,
        FormatOptions::default(),
        MetadataOptions::default(),
    )?;

    // Pick the default audio track and create a decoder for it. Gapless decoding is on by
    // default, so encoder delay and padding are removed.
    let track = format.default_track(TrackType::Audio).ok_or("no audio track")?;
    let track_id = track.id;
    let params = track.codec_params.as_ref().and_then(|params| params.audio()).ok_or("no codec")?;

    let mut decoder = mediaengine::default::get_codecs()
        .make_audio_decoder(params, &AudioDecoderOptions::default())?;

    // Read packets and decode those of the selected track.
    let mut frames = 0;

    while let Some(packet) = format.next_packet()? {
        if packet.track_id == track_id {
            frames += decoder.decode(&packet)?.frames();
        }
    }

    println!("decoded {frames} frames");
    Ok(())
}

The pieces are separate on purpose. A FormatReader (demuxer) yields packets with timestamps and trims, and a decoder turns them into audio buffers. FormatReader::seek positions a reader; after a seek, call reset() on the decoder and discard decoded frames until the requested timestamp. GETTING_STARTED.md walks through the whole API, and mediaengine/examples has runnable examples.

Tools

• mediaengine-play: probes, decodes, verifies and plays a file; it can seek, pick a track and turn gapless decoding off.
• mediaengine-check: compares decoded output against a reference decoder.

Development

The minimum supported Rust version is 1.85. Commands, conventions and known pitfalls are in AGENTS.md.

Credits

mediaengine is built on Symphonia, the pure-Rust audio decoding and media demuxing library by Philip Deljanov and the Symphonia contributors (see Symphonia's CONTRIBUTORS file). Most of the code here is theirs, and it keeps Symphonia's licence, the Mozilla Public License 2.0, along with its copyright notices. The AAC-LC decoder was originally written by Kostya Shishkov for the NihAV project and relicensed for Symphonia with his permission.

GETTING_STARTED.md is adapted from Symphonia's guide. Symphonia's benchmarks of its decoders against FFmpeg are in its BENCHMARKS.md.

License and Legal Information

This repository contains open-source code licensed under the Mozilla Public License 2.0. A copy of the license can be found in the repository's LICENSE file.

Please note: The Mozilla Public License 2.0 does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the Mozilla Public License 2.0 granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

Company Information

Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.