Skip to main content

MoQ media formats

Pure LOC (draft-ietf-moq-loc-04) and independent MSF (draft-ietf-moq-msf-01) format helpers. Depends only on the generic MoQ core. No transport libraries or Relay connection required. The one encoder, moq_media.opus, needs the opus extra (see Opus); import moq_media never loads it.

The catalog builder emits compact UTF-8 JSON bytes. It withholds avc1/avc3 until non-empty init data exists, derives their profile/constraint/level suffix from avcC (preserving avc3), forces Opus to 48 kHz, and rejects duplicate names and media roles on data tracks. Catalog namespaces use the player's slash-joined dialect, so fields containing / or empty fields are rejected. Source URLs can escape literal slashes in tuple fields. Omitted fields never become JSON nulls. Empty init data is omitted for every codec, and catalog strings must be valid Unicode (no lone surrogates).

LOC helpers accept nonnegative microsecond timestamps and a positive timescale, both safe integers (up to 2^53−1). A custom timescale rounds to the nearest tick, ties upward. Pass decoder configs only on keyframes, declare the same timescale on the track, and keep init data in the catalog too.

H.264 helpers split Annex B, build avcC from SPS/PPS, and convert access units to four-byte NAL length prefixes. Conversion removes SPS, PPS, and AUD by default, as avc1 requires. Codec-string inspection checks the avcC version/header; it does not validate SPS semantics or decode a bitstream. Construction from SPS/PPS supports Baseline, Main, and Extended profiles. Other profiles require extension fields and are rejected; callers can still supply an encoder-generated High-profile avcC to the catalog builder.

build_msf_catalog describes a live broadcast and leaves isComplete out: msf-01 §5.1.3 forbids the field while false, and once a catalog carries it, it can never be removed. When a broadcast ends on purpose, close the media tracks first (each close tells subscribers the track ended), then write build_msf_end_catalog(generated_at_ms=...) to the catalog track if it has demand, then close it (msf-01 §11.3). Do not send it after a crash or a lost connection.

parse_msf_catalog reads an independent catalog object's payload back into an MsfCatalog of MsfCatalogTrack descriptions: namespace tuple, name, packaging, live status, role, codec, dimensions, rates, bitrate, render group, and init data resolved from initRef to bytes. It raises ValueError on unsupported versions and delta updates, ignores unknown fields, and returns non-loc packagings (such as moq-rs cmaf) as declared.

For simulcast layers or a second source, pass media: a list of VideoTrackInfo and AudioTrackInfo, each with a render_group (default 1, tracks played together) and an optional alt_group (alternate versions of one content, of which a viewer plays one). Members of an alternate group share a role and a render group. parse_msf_catalog returns both fields, and group_msf_tracks(catalog.tracks) lists the sets a viewer picks one track from. See several media tracks.

catalog = build_msf_catalog(
    namespace,
    media=[
        VideoTrackInfo(name="video-hi", codec="vp8", width=1280, height=720, alt_group=1),
        VideoTrackInfo(name="video-lo", codec="vp8", width=320, height=180, alt_group=1),
        AudioTrackInfo(name="audio"),
        VideoTrackInfo(name="screen", codec="vp9", render_group=2),
    ],
)

Every media track shares one timeline: LOC timestamps are capture times in microseconds since the Unix epoch, with Timescale 1,000,000 declared on each track. BroadcastClock() pins a moment on the monotonic clock to the system clock (origin_us, origin_epoch_us) and counts from there. Use from_source_us for capture timestamps on that clock. Use source_timeline() for a clock with its own origin and rate, kept in line with calibrate(source_us, at_us=...) pairs. No browser or OS reports a microphone's input delay, so pass the one you know as capture_latency_us= and set timeline.capture_latency_us again when the device changes (about 30 ms for a laptop microphone, 190 ms for AirPods). Use sample_timeline(48_000) for a sample counter with nothing to calibrate against, such as a file. PlayoutClock() is the player side. Audio is the master: report what is audible with report_audio(timestamp_us), and due(timestamp_us) returns a PlayoutDecision to wait, show, or drop a video frame. With no audio it paces by wall clock. A broadcast with no video never reports a keyframe interval, so its clock would keep the 2 s default target latency, and a lasting delay increase would cost 2 s of silence before a frame sequencer resyncs: create an audio-only clock with PlayoutClock(target_latency_us=AUDIO_ONLY_TARGET_LATENCY_US) (500 ms), the same constant as TypeScript's. A clock nothing anchors, as in a bot that only decodes, judges nothing late and needs none. See spec/media-layer.md.

On the subscriber side, loc_timestamp_us(obj.properties, track_properties=ok.track_properties) reads an object's Timestamp in microseconds, applying whichever Timescale the object or the SUBSCRIBE_OK track properties declare.

msf-01's Media Timeline track holds one record per video group, mapping media time and the group's location to wall-clock time, for seeking and playlists. On the epoch timeline the two times are the same, so an entry needs only the timestamp and the group. Announce it with media_timeline=MediaTimelineTrackInfo() in build_msf_catalog. MediaTimeline keeps one broadcast's records, 512 by default, and returns the document to send each time a keyframe has opened a video group; caprid_relay.media.DocumentService sends it as a new timeline group, and again whenever the relay subscribes. See spec/media-layer.md.

timeline = MediaTimeline()
service = DocumentService(timeline_track)  # from caprid_relay.media; run service.serve() as a task
# Once the frame writer reports the keyframe that opened video group `group_id` as sent:
service.update(timeline.add(timestamp_us, group_id))
read = parse_media_timeline(timeline_object.payload)

Shared vectors live in spec/vectors/media.json and spec/vectors/clock.json. See UPSTREAM.json for Video.js provenance. Publishing handles and the object-to-frame reader are later stages of spec/media-layer.md.

from moq_media import (
    AudioTrackInfo,
    VideoTrackInfo,
    build_msf_catalog,
    loc_properties,
    loc_track_properties,
    parse_msf_catalog,
)
from moq_media.h264 import H264Config

h264 = H264Config.from_annexb(encoder.extradata)
catalog = build_msf_catalog(
    ["room", "live"],
    video=VideoTrackInfo(codec=h264.codec, width=640, height=360, init_data=h264.avcc),
    audio=AudioTrackInfo(),
)
video = publisher.register_track("video", track_properties=loc_track_properties())
# The application opens a group at each keyframe and handles demand/recovery.
await group.write(h264.to_avcc(access_unit), properties=loc_properties(timestamp_us, video_config=h264.avcc))

tracks = parse_msf_catalog(catalog_object.payload, catalog_namespace=["room", "live"]).tracks

Interrupts

A talker that stops mid-utterance, such as a voice bot the user talks over, writes an interrupt so listeners drop the audio they already hold. List the track in the catalog (data=[DataTrackInfo(name=INTERRUPT_TRACK)]), register it, and write build_interrupt(until_us=...) to it, with until_us the end of the last audio sent: the last OpusPacket's timestamp_us + duration_us. Start the next audio at or after it. parse_interrupt(payload) reads one back as an Interrupt. See spec/media-layer.md.

Transcripts

A talker publishes the text of its audio on the transcript track, so listeners show it as the audio plays. List the track (DataTrackInfo(name=TRANSCRIPT_TRACK)), register it, and write build_transcript_segment(seq=, from_us=, text=) just before the segment's first audio, then the same seq with until_us= and final=True once its last audio is sent; send no final version for a segment an interrupt cut. A listener reads each with parse_transcript_segment, keeps them in a Transcript (apply, and cut on each interrupt with the playout position), and shows lines_at(clock.position_us). See spec/media-layer.md.

Opus

pip install "caprid-moq-media[opus]" adds PyAV and moq_media.opus: Opus encoding and decoding for a server-side talker such as a voice bot. OpusEncoder takes interleaved s16 PCM (or a PyAV AudioFrame) at any rate and returns 20 ms OpusPackets stamped on the broadcast timeline. Its defaults are for voice: mono, 32 kb/s, libopus's voip application. Audio comes in runs. The first encode of a run gives the timestamp of its first sample, later calls continue by sample count, and flush() ends the run (an utterance) by padding the last partial frame with silence. OpusDecoder(rate=16_000) turns packets back into s16 PCM at the rate speech recognition wants. Call reset() at a frame's discontinuity.

from moq_media import AudioTrackInfo
from moq_media.opus import OpusDecoder, OpusEncoder

encoder = OpusEncoder(input_rate=24_000)  # the TTS engine's rate
audio_info = AudioTrackInfo(channel_config="1", bitrate=encoder.bitrate)  # for build_msf_catalog
# An OpusPacket carries its LOC properties, so caprid_relay.media.FrameWriter takes it as a frame.
for packet in encoder.encode(tts_pcm, timestamp_us=clock.now_us()) + encoder.flush():
    await frame_writer.write(packet)

decoder = OpusDecoder(rate=16_000)
async for frame in frame_reader:
    if frame.discontinuity:
        decoder.reset()
    stt.feed(decoder.decode(frame.payload))

Packet k of a run is stamped at the run's timestamp plus 20 ms per packet before it, as WebCodecs stamps its output; decoded audio trails the stamps by libopus's look-ahead: 6.5 ms, or 2.5 ms with application="lowdelay". A flushed run has one packet more than the frames it was given, which carries the look-ahead's tail. opus_packet_duration_us(payload) reads a packet's duration from its TOC byte without PyAV, held to the same vectors as TypeScript's opusPacketDurationUs. See ADR 0006.

The distribution on PyPI is caprid-moq-media; the import name is moq_media.

Stability

Before 1.0, a minor version can break the API. This package releases together with caprid-relay at the same version; see the changelog.

Metadata

Release files for caprid-moq-media 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for caprid-moq-media 0.1.0
File Size Uploaded Provenance Actions
caprid_moq_media-0.1.0.tar.gz 36.9 kB With provenance Details for caprid_moq_media-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for caprid-moq-media 0.1.0
File Interpreter ABI Platform Actions
caprid_moq_media-0.1.0-py3-none-any.whl Python 3 none any Details for caprid_moq_media-0.1.0-py3-none-any.whl

Total release size: 80.7 kB

Release files / caprid_moq_media-0.1.0.tar.gz

Download URL caprid_moq_media-0.1.0.tar.gz
Size 36.9 kB
Tags Source
SHA-256 checksum
How to use checksums
52dcbb2d164c298925b78d25ea7ac09591d7139b45009f2caa90e44759bd3b4a
BLAKE2b-256 checksum
How to use checksums
664baf11bad36e9ee142edc12a834dca6d052dd563e62d297ac26b588dd3c49b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Release files / caprid_moq_media-0.1.0-py3-none-any.whl

Download URL caprid_moq_media-0.1.0-py3-none-any.whl
Size 43.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
947d61418af3212a76a23cd2220755ac2d50cab78d423652d2f951d5716d179a
BLAKE2b-256 checksum
How to use checksums
df2b43ed1c903189cb1f16028d091bea07f4b065b7a831325682c8fd5c3b9dfd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Security for caprid-moq-media 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for caprid_moq_media-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page