TensorCodec

TorchCodec-style video and audio decoding, without PyTorch.

• Use the CPU decoder API and playback rules of TorchCodec 0.17.0.
• Get NumPy arrays instead of torch.Tensor.
• Install NumPy + TensorCodec. No Torch, PyAV, or FFmpeg CLI at runtime.

The goal is predictable frame selection, timestamps and audio ranges with a small runtime dependency set. This is a CPU decoding subset, not the entire TorchCodec package. It does not promise a speedup over PyAV or TorchCodec.

What is available?

Compatibility means

• Match the supported CPU API's frame selection, ordering, timing and metadata.
• Check behavior independently and against pinned TorchCodec 0.17.0.
• Allow color-conversion rounding: at most 1 uint8 unit or 1/65535 for float32 in the tested cases. Do not claim identical pixels across every FFmpeg build.
• Accept empty index lists, including the case affected by the reference's empty-list dtype inference bug.

Details and the tested scope: compatibility contract.

Current limits

• Binary wheels: Linux x86_64, glibc 2.28+, CPython 3.10+.
• No macOS or Windows wheels yet; free-threaded Python is not a release target.
• Exact video seeking scans packet timestamps when opening the decoder.
• Audio range queries currently decode from the beginning; late ranges can be expensive.
• NumPy return types require caller changes where code expects Torch tensors.
• Historical avdec benchmarks are not TensorCodec performance results.

Install

python -m pip install tensorcodec

Linux wheels bundle shared FFmpeg libraries. Source builds need Rust, libclang and FFmpeg 7 development headers/libraries.

Before the first PyPI upload, install a local wheel:

python -m pip install dist/tensorcodec-*.whl

Use

from tensorcodec.decoders import VideoDecoder, AudioDecoder

with VideoDecoder("video.mp4") as video:
    frame = video.get_frame_played_at(1.25)  # frame playing at this time
    print(frame.data.shape)                 # CHW NumPy array
    batch = video.get_frames_at([4, 0, 4])   # order and duplicates preserved
    clip = video.get_frames_played_in_range(0, 1, fps=8)

with AudioDecoder("audio.wav", sample_rate=16000, num_channels=1) as audio:
    samples = audio.get_samples_played_in_range(0, 1)
    print(samples.data.shape)               # channels × samples, float32

Decoded arrays keep their storage after the decoder closes. Input file objects remain caller-owned.

Implementation

A batch crosses the Python/Rust boundary once. Native decoding releases the GIL.

Development and verification

# Requires Rust, Clang/libclang, pkg-config and FFmpeg 7 development libraries.
python -m venv .venv
. .venv/bin/activate
python -m pip install numpy pytest ruff 'maturin>=1.8,<2'
maturin develop --locked

# Reference dependencies are for tests only.
python -m pip install torch==2.14.1 torchcodec==0.17.0 \
  --index-url https://download.pytorch.org/whl/cpu
pytest tests/test_video_contract.py tests/test_audio_contract.py --backend torchcodec
pytest --compare

Tests generate fixtures with FFmpeg/ffprobe and Python's wave module. --compare requires the exact oracle version; otherwise differential tests skip. The old avdec decoder and tests are never executed.

• Playback rules
• Release builds and PyPI publishing
• Native dependency licenses and source/build notices

TensorCodec's own code is MIT licensed.