Audio decoding

Use remedia-core to decode files, HTTP(S) URLs, owned bytes, or bounded producer streams into normalized PCM. Calls use native FFmpeg libraries and block the calling thread. Async applications should run them on blocking workers.

For Node.js input ownership, use Buffers, streams, and probing. For component boundaries, see Engine architecture.

Output formatCopy Anchor Link

Every PcmFrame contains owned interleaved f32 samples in the normalized format:

• 48,000 samples per second
• 2 channels in Front Left / Front Right order
• Interleaving L0, R0, L1, R1, ...
• Samples clamped to the conventional -1.0..=1.0 range. Non-finite values become zero

Mono sources are mapped to both stereo channels. Sources with more than two channels are downmixed by libswresample using the source channel layout. The decoder emits variable length frames and never pads them to a fixed duration or paces them in real time. The output layer can packetize these frames later.

PcmFrame.samples_per_channel is the number of output sample frames. Its samples vector contains twice that many scalar f32 values. start_sample is the frame's start position on the 48 kHz output timeline. The decoder's output_format() returns the shared PcmFormat::NORMALIZED constant.

For compatible Opus streams, playback may bypass PCM decoding. See Opus passthrough for eligibility and decoder warm-up. The AudioDecoder pull API always returns PCM.

Open and consume a fileCopy Anchor Link

Choose a source variant explicitly. A string is never guessed to be a file path or URL.

use remedia_core::{AudioDecoder, InputSource};

let mut decoder = AudioDecoder::open(InputSource::file("media/track.flac"))?;
let info = decoder.info();
println!("{}; {:?} Hz; {:?} channels", info.codec, info.sample_rate, info.channels);

let mut total_sample_frames = 0;
while let Some(frame) = decoder.next_frame()? {
    // Read or copy frame.samples before requesting the next block.
    total_sample_frames += frame.samples_per_channel;
}
println!("Decoded {total_sample_frames} sample frames per channel");
# Ok::<(), Box<dyn std::error::Error>>(())

Use InputSource::http_url(url) for http:// or https:// sources. HTTP redirects and responses without a known content length are handled by FFmpeg's demuxer. HTTP status failures, network timeouts, invalid media, and missing audio streams return errors. The server must expose a supported FFmpeg HTTP(S) protocol and the requested media must be reachable from the process.

Single-use HTTP resourcesCopy Anchor Link

FFmpeg's HTTP protocol asks for Range: bytes=0- by default and reopens the resource to seek or reconnect. A resource which can be read exactly once, such as a temporary media grant, answers that request with HTTP 416. Use InputSource::sequential_http_url(url) for these resources:

use remedia_core::{AudioDecoder, InputSource};

let mut decoder =
    AudioDecoder::open(InputSource::sequential_http_url("https://media.example.com/grant/token"))?;
while let Some(frame) = decoder.next_frame()? {
    // Consume the frame before requesting the next one.
    let _ = frame.samples_per_channel;
}
# Ok::<(), Box<dyn std::error::Error>>(())

The input sends one plain GET, disables FFmpeg seeking and automatic reconnection, and restricts redirects to HTTP(S) transports. It is a forward-only byte stream, like InputSource::byte_stream: seek can move forward by decoding and discarding frames, and seeking backwards returns DecoderError::UnsupportedSeeking. An interrupted transfer is reported rather than resumed from another offset, and the source cannot be reopened. Resolve a fresh resource instead. InputSource::http_url keeps FFmpeg's seekable behavior and is not changed. Diagnostics report this source kind as http_sequential. Callers accepting untrusted URLs must apply address, redirect, and size policy. The server plugin-media path applies these restrictions before feeding its bounded input.

In-memory bytes and producer streamsCopy Anchor Link

InputSource::bytes owns encoded bytes and permits byte-offset seeking when the container supports it. Cloning the source shares its owned byte allocation:

use remedia_core::{AudioDecoder, InputSource};

let bytes = std::fs::read("media/track.wav")?;
let mut decoder = AudioDecoder::open(InputSource::bytes(bytes))?;
while let Some(frame) = decoder.next_frame()? {
    consume(frame.samples);
}
# fn consume(_: Vec<f32>) {}
# Ok::<(), Box<dyn std::error::Error>>(())

For incremental producers, InputSource::byte_stream(max_pending_bytes) returns the source and a ByteStreamWriter. Writes block when the pending byte capacity is reached, applying backpressure without dropping media bytes. Signal normal EOF with finish() or propagate a producer failure with fail(message). Dropping/cancelling the decoder wakes blocked reads and writes and clears queued bytes. A producer-backed input is one-shot and non-seekable:

use remedia_core::{AudioDecoder, InputSource};

let (source, writer) = InputSource::byte_stream(512 * 1024)?;
let producer = std::thread::spawn(move || -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    for encoded_chunk in encoded_chunks() {
        writer.write(&encoded_chunk)?;
    }
    writer.finish()?;
    Ok(())
});

let mut decoder = AudioDecoder::open(source)?;
while let Some(frame) = decoder.next_frame()? {
    consume(frame.samples);
}
producer.join().expect("producer thread panicked")?;
# fn encoded_chunks() -> Vec<Vec<u8>> { Vec::new() }
# fn consume(_: Vec<f32>) {}
# Ok::<(), Box<dyn std::error::Error>>(())

The queue capacity limits bytes waiting inside Remedia. Producers may hold their current chunk and their own upstream buffers, so they should also use bounded buffers. For Node.js Readables and the JavaScript ownership rules, see Node.js bindings.

MetadataCopy Anchor Link

decoder.info() returns AudioStreamInfo for the selected audio stream:

• stream_index and codec
• Optional source sample_rate, channel count, and channel-layout description
• Optional media duration and bit rate
• Available container and stream tags in metadata

Duration and some stream properties can be unknown, especially for live or incomplete sources. AudioDecoder::open selects FFmpeg's best usable audio stream, so a video stream before the audio stream does not prevent decoding. Inputs without a usable audio stream return DecoderError::NoAudioStream.

HTTP timeouts and cancellationCopy Anchor Link

Network reads have a 15-second I/O timeout by default. Set another timeout with DecoderOptions::io_timeout. To interrupt a blocked open or read, pass a cloneable cancellation token before opening the source:

use std::time::Duration;
use remedia_core::{AudioDecoder, CancellationToken, DecoderOptions, InputSource};

let cancellation = CancellationToken::new();
let options = DecoderOptions {
    io_timeout: Duration::from_secs(8),
    cancellation: cancellation.clone(),
};
let mut decoder = AudioDecoder::open_with_options(
    InputSource::http_url("https://example.com/audio.mp3"),
    options,
)?;

// Another thread can call cancellation.cancel() to interrupt a blocked read.
let mut total_sample_frames = 0;
while let Some(frame) = decoder.next_frame()? {
    total_sample_frames += frame.samples_per_channel;
}
println!("Decoded {total_sample_frames} sample frames per channel");
# Ok::<(), Box<dyn std::error::Error>>(())

Each decoder owns its demuxer, decoder, and resampler, so independent sessions do not share playback state. The pull API applies backpressure naturally: the next input packet is read only as the caller requests more frames. Drop the decoder to close its native resources.

Example and current limitsCopy Anchor Link

For playback state, output consumers, pause/resume, session seeking, and cleanup, see the playback session guide.

Run the example from the repository root:

cargo run -p remedia-core --example decode -- file ./media/track.flac
cargo run -p remedia-core --example decode -- url https://example.com/audio.mp3

The example prints source metadata and the decoded sample count and duration. The decoder supports seeking when the input demuxer can seek. Playback sessions and generic PCM outputs are documented separately. Device output and remote stream uploads are not implemented. Filters and Opus encoding are available through the core adapters described in the playback guide. Supported codecs and containers depend on the native FFmpeg libraries installed on the system.

Disposable metadata probingCopy Anchor Link

probe(source: InputSource, options: ProbeOptions) -> Result<AudioStreamInfo, DecoderError> opens an input and runs native FFmpeg stream discovery, then closes it immediately. It creates no playback session, PCM decoder, filtergraph, output, or voice transport. Container tags are merged with selected audio stream tags. Stream tags take precedence. Unknown fields stay None.

use remedia_core::{probe, InputSource, ProbeOptions};
let info = probe(InputSource::file("audio.mp3"), ProbeOptions::default())?;

This is blocking I/O: use a blocking worker in asynchronous applications. ProbeOptions.timeout is a positive total deadline (15 seconds by default); cancellation interrupts native I/O and bounded byte-stream reads. Errors use DecoderError, including InputTimeout, Cancelled, invalid media and NoAudioStream. All core input kinds are supported. Stream inputs are non-seekable and disposable. Probing may consume some or all bytes. No source is retained for playback. FFmpeg may inspect packets while discovering formats, but Remedia does not run a PCM decoding loop.

Linked FFmpeg runtime versionsCopy Anchor Link

remedia_core::ffmpeg_runtime_versions() returns an array of five (&'static str, u32) pairs for avcodec, avformat, avutil, avfilter, and swresample. Each u32 is the runtime library's packed version number: major << 16 | minor << 8 | micro. The call is read-only, does not initialize playback, and does not require a media source.

for (library, version) in remedia_core::ffmpeg_runtime_versions() {
    println!("{library}: ABI major {}", version >> 16);
}

remedia-server doctor uses this to check the active FFmpeg 9 ABI majors. A missing linked library may prevent the executable from starting, before this function can run. See Install FFmpeg 9.