Skip to main content

MoQ media formats

Pure LOC (draft-ietf-moq-loc-04) and independent MSF (draft-ietf-moq-msf-01) format helpers. The package depends only on the generic MoQ core, and needs no transport library or Relay connection. The one encoder, moq_media.opus, needs the opus extra (see Opus). import moq_media never loads it.

The catalog builder emits compact UTF-8 JSON bytes. It withholds avc1 and avc3 until non-empty init data exists, derives their profile, constraint, and level suffix from avcC (keeping avc3), forces Opus to 48 kHz, and rejects duplicate names and media roles on data tracks. Catalog namespaces use the player's slash-joined dialect, so fields containing / or empty fields are rejected. Source URLs can escape literal slashes in tuple fields. Omitted fields never become JSON nulls. Empty init data is omitted for every codec, and catalog strings must be valid Unicode (no lone surrogates).

LOC helpers accept nonnegative microsecond timestamps and a positive timescale, both safe integers (up to 2^53−1). A custom timescale rounds to the nearest tick, ties upward. Pass decoder configs only on keyframes, declare the same timescale on the track, and keep init data in the catalog too.

H.264 helpers split Annex B, build avcC from SPS/PPS, and convert access units to four-byte NAL length prefixes. Conversion removes SPS, PPS, and AUD by default, as avc1 requires. Codec-string inspection checks the avcC version and header. It does not validate SPS semantics or decode a bitstream. Construction from SPS/PPS supports Baseline, Main, and Extended profiles. Other profiles require extension fields and are rejected. Callers can still supply an encoder-generated High-profile avcC to the catalog builder.

build_msf_catalog describes a live broadcast and leaves isComplete out: msf-01 §5.1.3 forbids the field while false, and once a catalog carries it, it can never be removed. When a broadcast ends on purpose, close the media tracks first (each close tells subscribers the track ended), then write build_msf_end_catalog(generated_at_ms=...) to the catalog track if it has demand, then close it (msf-01 §11.3). Do not send it after a crash or a lost connection.

parse_msf_catalog reads an independent catalog object's payload back into an MsfCatalog of MsfCatalogTrack descriptions: namespace tuple, name, packaging, live status, role, codec, dimensions, rates, bitrate, render group, and init data resolved from initRef to bytes. It raises ValueError on unsupported versions and delta updates, ignores unknown fields, and returns non-loc packagings (such as moq-rs cmaf) as declared.

For simulcast layers or a second source, pass media: a list of VideoTrackInfo and AudioTrackInfo, each with a render_group (default 1, tracks played together) and an optional alt_group (alternate versions of one content, of which a viewer plays one). Members of an alternate group share a role and a render group. parse_msf_catalog returns both fields, and group_msf_tracks(catalog.tracks) lists the sets a viewer picks one track from. See several media tracks.

catalog = build_msf_catalog(
    namespace,
    media=[
        VideoTrackInfo(name="video-hi", codec="vp8", width=1280, height=720, alt_group=1),
        VideoTrackInfo(name="video-lo", codec="vp8", width=320, height=180, alt_group=1),
        AudioTrackInfo(name="audio"),
        VideoTrackInfo(name="screen", codec="vp9", render_group=2),
    ],
)

Every media track shares one timeline: LOC timestamps are capture times in microseconds since the Unix epoch, with Timescale 1,000,000 declared on each track. BroadcastClock() pins a moment on the monotonic clock to the system clock (origin_us, origin_epoch_us) and counts from there. Use from_source_us for capture timestamps on that clock. Use source_timeline() for a clock with its own origin and rate, kept in line with calibrate(source_us, at_us=...) pairs. No browser or OS reports a microphone's input delay, so pass the one you know as capture_latency_us= and set timeline.capture_latency_us again when the device changes (about 30 ms for a laptop microphone, 190 ms for AirPods). Use sample_timeline(48_000) for a sample counter with nothing to calibrate against, such as a file. PlayoutClock() is the player side. Audio is the master: report what is audible with report_audio(timestamp_us), and due(timestamp_us) returns a PlayoutDecision to wait, show, or drop a video frame. With no audio it paces by wall clock. A broadcast with no video never reports a keyframe interval, so its clock would keep the 2 s default target latency. A lasting delay increase would then cost 2 s of silence before a frame sequencer resyncs. Create an audio-only clock with PlayoutClock(target_latency_us=AUDIO_ONLY_TARGET_LATENCY_US) (500 ms), the same constant as TypeScript's. A clock nothing anchors, as in a bot that only decodes, judges nothing late and needs none. See spec/media-layer.md.

On the subscriber side, loc_timestamp_us(obj.properties, track_properties=ok.track_properties) reads an object's Timestamp in microseconds, applying whichever Timescale the object or the SUBSCRIBE_OK track properties declare.

msf-01's Media Timeline track holds one record per video group, mapping media time and the group's location to wall-clock time, for seeking and playlists. On the epoch timeline the two times are the same, so an entry needs only the timestamp and the group. Announce it with media_timeline=MediaTimelineTrackInfo() in build_msf_catalog. MediaTimeline keeps one broadcast's records, 512 by default, and returns the document to send each time a keyframe has opened a video group. caprid_relay.media.DocumentService sends it as a new timeline group, and again whenever the relay subscribes. See spec/media-layer.md.

timeline = MediaTimeline()
service = DocumentService(timeline_track)  # from caprid_relay.media; run service.serve() as a task
# Once the frame writer reports the keyframe that opened video group `group_id` as sent:
service.update(timeline.add(timestamp_us, group_id))
read = parse_media_timeline(timeline_object.payload)

Shared vectors live in spec/vectors/media.json and spec/vectors/clock.json. See UPSTREAM.json for Video.js provenance. The publishing handles (FrameWriter, CatalogService, DocumentService) and the object-to-frame reader (FrameReader) need a Relay connection, so they live in caprid_relay.media. FrameSequencer here is the reader's transport-free engine: it puts one track's objects in decode order and drops what is too late against a PlayoutClock.

from moq_media import (
    AudioTrackInfo,
    VideoTrackInfo,
    build_msf_catalog,
    loc_properties,
    loc_track_properties,
    parse_msf_catalog,
)
from moq_media.h264 import H264Config

h264 = H264Config.from_annexb(encoder.extradata)
catalog = build_msf_catalog(
    ["room", "live"],
    video=VideoTrackInfo(codec=h264.codec, width=640, height=360, init_data=h264.avcc),
    audio=AudioTrackInfo(),
)
video = publisher.register_track("video", track_properties=loc_track_properties())
# The application opens a group at each keyframe and handles demand and recovery.
await group.write(h264.to_avcc(access_unit), properties=loc_properties(timestamp_us, video_config=h264.avcc))

tracks = parse_msf_catalog(catalog_object.payload, catalog_namespace=["room", "live"]).tracks

Interrupts

A talker that stops mid-utterance, such as a voice bot the user talks over, writes an interrupt so listeners drop the audio they already hold. List the track in the catalog (data=[DataTrackInfo(name=INTERRUPT_TRACK)]), register it, and write build_interrupt(until_us=...) to it, with until_us the end of the last audio sent: the last OpusPacket's timestamp_us + duration_us. Start the next audio at or after it. parse_interrupt(payload) reads one back as an Interrupt. See spec/media-layer.md.

Transcripts

A talker publishes the text of its audio on the transcript track, so listeners show it as the audio plays. List the track (DataTrackInfo(name=TRANSCRIPT_TRACK)), register it, and write build_transcript_segment(seq=, from_us=, text=) just before the segment's first audio, then the same seq with until_us= and final=True once its last audio is sent. Send no final version for a segment an interrupt cut. A listener reads each with parse_transcript_segment, keeps them in a Transcript (apply, and cut on each interrupt with the playout position), and shows lines_at(clock.position_us). See spec/media-layer.md.

Opus

pip install "caprid-moq-media[opus]" adds PyAV and moq_media.opus: Opus encoding and decoding for a server-side talker such as a voice bot. OpusEncoder takes interleaved s16 PCM (or a PyAV AudioFrame) at any rate and returns 20 ms OpusPackets stamped on the broadcast timeline. Its defaults are for voice: mono, 32 kb/s, libopus's voip application. Audio comes in runs. The first encode of a run gives the timestamp of its first sample, later calls continue by sample count, and flush() ends the run (an utterance) by padding the last partial frame with silence. OpusDecoder(rate=16_000) turns packets back into s16 PCM at the rate speech recognition wants. Call reset() at a frame's discontinuity.

from moq_media import AudioTrackInfo
from moq_media.opus import OpusDecoder, OpusEncoder

encoder = OpusEncoder(input_rate=24_000)  # the TTS engine's rate
audio_info = AudioTrackInfo(channel_config="1", bitrate=encoder.bitrate)  # for build_msf_catalog
# An OpusPacket carries its LOC properties, so caprid_relay.media.FrameWriter takes it as a frame.
for packet in encoder.encode(tts_pcm, timestamp_us=clock.now_us()) + encoder.flush():
    await frame_writer.write(packet)

decoder = OpusDecoder(rate=16_000)
async for frame in frame_reader:
    if frame.discontinuity:
        decoder.reset()
    stt.feed(decoder.decode(frame.payload))

Packet k of a run is stamped at the run's timestamp plus 20 ms per packet before it, as WebCodecs stamps its output. Decoded audio trails the stamps by libopus's look-ahead, which is 6.5 ms, or 2.5 ms with application="lowdelay". A flushed run has one packet more than the frames it was given, which carries the look-ahead's tail. opus_packet_duration_us(payload) reads a packet's duration from its TOC byte without PyAV, held to the same vectors as TypeScript's opusPacketDurationUs. See ADR 0006.

The distribution on PyPI is caprid-moq-media, and the import name is moq_media.

Stability

Before 1.0, a minor version can break the API. This package releases together with caprid-relay at the same version. See the changelog.

Metadata

Release files for caprid-moq-media 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for caprid-moq-media 0.3.0
File Size Uploaded Provenance Actions
caprid_moq_media-0.3.0.tar.gz 37.3 kB With provenance Details for caprid_moq_media-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for caprid-moq-media 0.3.0
File Interpreter ABI Platform Actions
caprid_moq_media-0.3.0-py3-none-any.whl Python 3 none any Details for caprid_moq_media-0.3.0-py3-none-any.whl

Total release size: 81.6 kB

Release files / caprid_moq_media-0.3.0.tar.gz

Download URL caprid_moq_media-0.3.0.tar.gz
Size 37.3 kB
Tags Source
SHA-256 checksum
How to use checksums
10cc0d6b82f1bb49253761b90237f83fa84333fae19a0de301af83d96bd5a2b3
BLAKE2b-256 checksum
How to use checksums
9642f1ba69c74aa6040fcec80330e97b0fce1c8b381855ef0f331671a85388a1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 31, 2026.

Transparency log

Release files / caprid_moq_media-0.3.0-py3-none-any.whl

Download URL caprid_moq_media-0.3.0-py3-none-any.whl
Size 44.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
80df28386f3db0d14ecc2050dc9c67ca62e72a24b33392a52b645ea7e0622a07
BLAKE2b-256 checksum
How to use checksums
8649e8a8fce8200779f261f124175b2c0131542a4cdb9eaa4035ecacc8a26d45
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 31, 2026.

Transparency log

Security for caprid-moq-media 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for caprid_moq_media-0.3.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 31, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page