Skip to main content

AIND Video Utils

CI License ruff uv

Tools for working with video files using ffmpeg.

Prerequisites

This package requires ffmpeg and ffprobe to be installed and available on your PATH.

  • Linux: sudo apt install ffmpeg (Debian/Ubuntu) or sudo dnf install ffmpeg (Fedora)
  • macOS: brew install ffmpeg
  • Windows: Download from ffmpeg.org and add to PATH

Installation

# uv
uv add aind-video-utils                        # core only
uv add "aind-video-utils[transcode]"            # with transcode CLI
uv add "aind-video-utils[plotting]"             # with QC plotting

# pip
pip install aind-video-utils
pip install "aind-video-utils[transcode]"
pip install "aind-video-utils[plotting]"
pip install "aind-video-utils[parquet]"
Extra Adds
transcode pydantic-settings, rich — required for aind-transcode CLI
plotting matplotlib, opencv — required for aind-video-qc CLI
parquet pyarrow — required to write preview_metadata.parquet

Usage

from aind_video_utils import probe, extract_luma_frame, extract_srgb_frame

# Probe video metadata
info = probe("video.mp4")
print(info["streams"][0]["pix_fmt"])  # e.g. "yuv420p", "gbrp"

# Extract luma (Y) plane at t=1.0s
luma, color_range, bit_depth = extract_luma_frame("video.mp4", 1.0)

# Extract an sRGB frame at t=1.0s
srgb = extract_srgb_frame("video.mp4", 1.0)

Encoding Profiles

This package is the canonical Python source for the AIND behavior video file standard encoding profiles. SPEC_VERSION names the revision of the standard they implement. Four profiles are provided as frozen dataclass constants:

Constant Codec Pixel Format Container Use Case
OFFLINE_8BIT libx264 yuv420p mp4 Long-term storage (default)
OFFLINE_10BIT libx264 yuv420p10le mp4 Long-term storage, 10-bit
ONLINE_8BIT h264_nvenc yuv420p mkv Real-time acquisition
ONLINE_10BIT hevc_nvenc p010le mkv Real-time acquisition, 10-bit
from aind_video_utils import OFFLINE_8BIT, ONLINE_8BIT

# Inspect the exact ffmpeg args
OFFLINE_8BIT.ffmpeg_output_args()
ONLINE_8BIT.ffmpeg_input_args()

# Customize with replace()
fast = OFFLINE_8BIT.replace(codec_params=("-preset", "veryfast", "-crf", "18"))

For online acquisition pipelines that build their own ffmpeg command:

from aind_video_utils import ONLINE_8BIT

profile = ONLINE_8BIT
cmd = [
    "ffmpeg",
    *profile.ffmpeg_input_args(),
    "-f", "rawvideo", "-pix_fmt", "bgr24",
    "-s", f"{w}x{h}", "-r", str(fps),
    "-i", "pipe:0",
    *profile.ffmpeg_output_args(),
    str(output_path),
]

Transcoding Python API

from aind_video_utils import transcode_video, OFFLINE_8BIT

# Simplest form — offline 8-bit with automatic colorspace fix
transcode_video(input_path, output_path)

# Explicit profile
transcode_video(input_path, output_path, profile=OFFLINE_8BIT)

# Custom profile with speed override
fast = OFFLINE_8BIT.replace(codec_params=("-preset", "veryfast", "-crf", "18"))
transcode_video(input_path, output_path, profile=fast)

# Skip automatic setparams probing
transcode_video(input_path, output_path, auto_fix_colorspace=False)

# Legacy h264-in-AVI sources: re-stamp timestamps so ffmpeg keeps every frame
transcode_video(input_path, output_path, normalize_cfr=True)

# Hold the decoder, filters and each encoder to 8 threads, e.g. in an 8-CPU job
transcode_video(input_path, output_path, threads=8)

transcode_video() raises RuntimeError when the output holds a different number of frames than ffmpeg decoded from the source; pass fail_on_frame_drop=False to accept such an output.

Preview and Poster Derivatives

transcode_video() can also write a browser-playable preview and a JPEG poster, as extra outputs of the same ffmpeg process, under the names the standard gives them:

from aind_video_utils import transcode_video

transcode_video(
    input_path,
    camera_dir / "video.mp4",                  # archival, frame-exact
    preview_fps=30.0,                          # preview.mp4: every N-th source frame
    metadata_csv=camera_dir / "metadata.csv",  # preview_metadata.parquet: ReferenceTime of each preview frame
    poster=True,                               # poster.jpg: sRGB still of the middle frame
)

The poster's middle frame comes from the frame count the container records. Matroska records none, so for an .mkv it is estimated from the duration; count_frames=True counts packets for the exact frame instead, at the cost of reading the whole source once more, which for a URL is a second download. A recording killed mid-write records no duration either, and its packets are counted regardless. If a truncated file's duration places the poster past its end, ffmpeg fails the run, though the archival video it wrote is complete.

The standard requires preview_metadata.parquet beside every preview. Its row k is row kN of metadata.csv, so metadata.csv must have one row per frame of the archival video; transcode_video() raises otherwise and writes no sidecar. It needs the parquet extra. write_preview_metadata() writes one on its own.

The derivatives take these names when the archival video is named video.<ext>, as the standard names it. Otherwise they take its stem, so BottomCamera.mp4 gets BottomCamera_preview.mp4, BottomCamera_preview_metadata.parquet and BottomCamera_poster.jpg, and two videos in one folder keep theirs apart.

N puts the preview between 25 and 35 fps, preferring a whole-number rate:

Source N Preview
1000 fps 40 25 fps
500 fps 20 25 fps
300 fps 10 30 fps
240 fps 8 30 fps
120 fps 4 30 fps
29.97 fps 1 29.97 fps

The poster branches off the source ahead of the BT.709 chain, because browsers read JPEG as sRGB. The behavior video standard gives the rationale for both.

To build the command yourself, attach derivatives to a conditioned profile. Without with_setparams, the poster's zscale has no transfer function to resolve, and its failure takes the archival encode down with it. The poster's middle frame comes from the probe's frame count, or its duration when it has none; probe(input_path, count_packets=True) gives an .mkv an exact count. Pass filename=None to with_preview or with_poster to name a derivative from the primary's stem and a suffix instead, as in clip_preview.mp4:

from aind_video_utils import (
    OFFLINE_8BIT, probe, with_poster, with_preview, with_setparams,
)

probe_json = probe(input_path)
conditioned = with_setparams(OFFLINE_8BIT, probe_json)
profile = with_poster(with_preview(conditioned, probe_json), probe_json)

profile.output_paths(output_path)  # [video.mp4, preview.mp4, poster.jpg]
profile.ffmpeg_graph_args()        # ["-filter_complex", "[0:v]setparams=...,split=2[chain][d1];..."]
profile.ffmpeg_output_groups()     # one argument list per output, in the same order

ffmpeg_output_args() raises for a profile with derivatives, since one -vf cannot branch. The aind-transcode CLI does not expose derivatives yet.

Transcode CLI

With the transcode extra installed, the aind-transcode command is available:

aind-transcode videos/                                # defaults: offline-8bit, auto-fix
aind-transcode videos/ --profile offline-10bit        # explicit profile
aind-transcode videos/ --preset veryfast              # override speed
aind-transcode videos/ --crf 20 --preset veryfast     # override quality + speed
aind-transcode videos/ --no_auto_fix_colorspace true  # skip setparams probing
aind-transcode videos/ --normalize_cfr true           # legacy h264-in-AVI sources
aind-transcode videos/ --jobs 4                       # parallel workers

Settings can also be stored in aind-transcode.toml in the working directory:

input = ["videos/"]
output_dir = "transcoded"
profile = "offline-8bit"
overwrite = false

QC CLI

With the plotting extra installed, the aind-video-qc command is available:

# Compare linear-light input against BT.709-encoded output
aind-video-qc linear-to-bt709 input.mp4 output.mp4

# Compare ffmpeg luma extraction with OpenCV decode
aind-video-qc opencv input.mp4

# Options
aind-video-qc linear-to-bt709 input.mp4 output.mp4 --frame-time 1.5 --dpi 300 -o qc.png

QC Python API

from aind_video_utils.video_qc import compare_linear_to_bt709

fig = compare_linear_to_bt709("input.mp4", "output.mp4", frame_time=0)
fig.savefig("qc.png")

Contributing

See CONTRIBUTING.md for setup, checks and conventions.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Metadata

Release files for aind-video-utils 0.9.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aind-video-utils 0.9.0
File Size Uploaded Provenance Actions
aind_video_utils-0.9.0.tar.gz 254.2 kB Without provenance Details for aind_video_utils-0.9.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aind-video-utils 0.9.0
File Interpreter ABI Platform Actions
aind_video_utils-0.9.0-py3-none-any.whl Python 3 none any Details for aind_video_utils-0.9.0-py3-none-any.whl

Total release size: 326.6 kB

Release files / aind_video_utils-0.9.0.tar.gz

Download URL aind_video_utils-0.9.0.tar.gz
Size 254.2 kB
Tags Source
SHA-256 checksum
How to use checksums
8f7dbd7c91c8daa88d2610849f00b65b677c6636f7cbdacc4e284f1770ab2c61
BLAKE2b-256 checksum
How to use checksums
bce70e501ca18f296354763113a167a3c00dade08417d9208166fbed53d83049
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / aind_video_utils-0.9.0-py3-none-any.whl

Download URL aind_video_utils-0.9.0-py3-none-any.whl
Size 72.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d6de577366660e92a559abde647453aaf939ea918d64e616e2fd0f7b11bcac08
BLAKE2b-256 checksum
How to use checksums
f8a4d70c2f78759328cab7b3d81cb89605ddd35af1d4afc5db3acd9345d7afb3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for aind-video-utils 0.9.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page