TensorCodec

TorchCodec-style video and audio decoding, without PyTorch.

• Use the CPU decoder API and playback rules of TorchCodec 0.17.0.
• Get NumPy arrays instead of torch.Tensor.
• Install NumPy + TensorCodec. No Torch, PyAV, or FFmpeg CLI at runtime.

The goal is predictable frame selection, timestamps and audio ranges with a small runtime dependency set. This is a CPU decoding subset, not the entire TorchCodec package. It does not promise a speedup over PyAV or TorchCodec.

Scope compared with TorchCodec

TorchCodec includes decoders, encoders, sampling and transforms.

Legend for both tables: ✓ supported · △ limited support · — unavailable.

FPS-based decoder queries are available; clip samplers are not implemented.

Decoder compatibility

Compatibility means

• Match the supported CPU API's frame selection, ordering, timing and metadata.
• Check behavior independently and against pinned TorchCodec 0.17.0.
• Allow color-conversion rounding: at most 1 uint8 unit or 1/65535 for float32 in the tested cases. Do not claim identical pixels across every FFmpeg build.
• Accept empty index lists, including the case affected by the reference's empty-list dtype inference bug.

Details and the tested scope: compatibility contract.

Current limits

• Binary wheels: Linux x86_64 / ARM64 (aarch64), glibc 2.17+, CPython 3.10+.
• A compatible NumPy wheel is also required. On older glibc, the installer may select an older NumPy; newer Python versions may require a newer glibc.
• No macOS, Windows or musl/Alpine wheels yet; free-threaded Python is not a release target.
• Exact video seeking scans packet timestamps when opening the decoder.
• Seeking trusts container keyframe flags; incorrect flags can corrupt decoded frames. Corrected frame mappings or a repaired input are needed in that case.
• Audio range queries currently decode from the beginning; late ranges can be expensive.
• NumPy return types require caller changes where code expects Torch tensors.
• Historical avdec benchmarks are not TensorCodec performance results.

Install

uv pip install tensorcodec

Use an existing virtual environment, or create one with uv venv first. Linux wheels bundle shared FFmpeg libraries. Source builds need Rust, libclang and FFmpeg 7 development headers/libraries.

Use

from tensorcodec.decoders import VideoDecoder, AudioDecoder

with VideoDecoder("video.mp4") as video:
    frame = video.get_frame_played_at(1.25)  # frame playing at this time
    print(frame.data.shape)                 # CHW NumPy array
    batch = video.get_frames_at([4, 0, 4])   # order and duplicates preserved
    clip = video.get_frames_played_in_range(0, 1, fps=8)

with AudioDecoder("audio.wav", sample_rate=16000, num_channels=1) as audio:
    samples = audio.get_samples_played_in_range(0, 1)
    print(samples.data.shape)               # channels × samples, float32

Decoded arrays keep their storage after the decoder closes. Input file objects remain caller-owned.

Implementation

A batch crosses the Python/Rust boundary once. Native decoding releases the GIL.

Development and verification

# Requires uv, Rust, Clang/libclang, pkg-config and FFmpeg 7 development libraries.
# Build the editable package and install development + pinned CPU oracle groups.
uv sync --group dev --group oracle

uv run --group oracle pytest tests/test_video_contract.py tests/test_audio_contract.py --backend torchcodec
uv run --group oracle pytest --compare

# Rebuild after changing Rust code.
uv run --group oracle maturin develop --locked --uv

Tests generate fixtures with FFmpeg/ffprobe and Python's wave module. --compare requires the exact oracle version; otherwise differential tests skip. The old avdec decoder and tests are never executed.

• Playback rules
• Release builds and PyPI publishing
• Native dependency licenses and source/build notices

TensorCodec's own code is MIT licensed.