Skip to main content

AIND Video Utils

CI License ruff uv

Tools for working with video files using ffmpeg.

Prerequisites

This package requires ffmpeg and ffprobe to be installed and available on your PATH.

  • Linux: sudo apt install ffmpeg (Debian/Ubuntu) or sudo dnf install ffmpeg (Fedora)
  • macOS: brew install ffmpeg
  • Windows: Download from ffmpeg.org and add to PATH

Installation

# uv
uv add aind-video-utils                        # core only
uv add "aind-video-utils[transcode]"            # with transcode CLI
uv add "aind-video-utils[plotting]"             # with QC plotting

# pip
pip install aind-video-utils
pip install "aind-video-utils[transcode]"
pip install "aind-video-utils[plotting]"
pip install "aind-video-utils[parquet]"
Extra Adds
transcode pydantic-settings, rich — required for aind-transcode CLI
plotting matplotlib, opencv — required for aind-video-qc CLI
parquet pyarrow — required to write preview_metadata.parquet

Usage

from aind_video_utils import probe, extract_luma_frame, extract_srgb_frame

# Probe video metadata
info = probe("video.mp4")
print(info["streams"][0]["pix_fmt"])  # e.g. "yuv420p", "gbrp"

# Extract luma (Y) plane at t=1.0s
luma, color_range, bit_depth = extract_luma_frame("video.mp4", 1.0)

# Extract an sRGB frame at t=1.0s
srgb = extract_srgb_frame("video.mp4", 1.0)

Encoding Profiles

This package is the canonical Python source for the AIND behavior video file standard encoding profiles. SPEC_VERSION names the revision of the standard they implement. Four profiles are provided as frozen dataclass constants:

Constant Codec Pixel Format Container Use Case
OFFLINE_8BIT libx264 yuv420p mp4 Long-term storage (default)
OFFLINE_10BIT libx264 yuv420p10le mp4 Long-term storage, 10-bit
ONLINE_8BIT h264_nvenc yuv420p mkv Real-time acquisition
ONLINE_10BIT hevc_nvenc p010le mkv Real-time acquisition, 10-bit
from aind_video_utils import OFFLINE_8BIT, ONLINE_8BIT

# Inspect the exact ffmpeg args
OFFLINE_8BIT.ffmpeg_output_args()
ONLINE_8BIT.ffmpeg_input_args()

# Customize with replace()
fast = OFFLINE_8BIT.replace(codec_params=("-preset", "veryfast", "-crf", "18"))

For online acquisition pipelines that build their own ffmpeg command:

from aind_video_utils import ONLINE_8BIT

profile = ONLINE_8BIT
cmd = [
    "ffmpeg",
    *profile.ffmpeg_input_args(),
    "-f", "rawvideo", "-pix_fmt", "bgr24",
    "-s", f"{w}x{h}", "-r", str(fps),
    "-i", "pipe:0",
    *profile.ffmpeg_output_args(),
    str(output_path),
]

Transcoding Python API

from fractions import Fraction

from aind_video_utils import transcode_video, OFFLINE_8BIT

# Simplest form — offline 8-bit with automatic colorspace fix
transcode_video(input_path, output_path)

# Explicit profile
transcode_video(input_path, output_path, profile=OFFLINE_8BIT)

# Custom profile with speed override
fast = OFFLINE_8BIT.replace(codec_params=("-preset", "veryfast", "-crf", "18"))
transcode_video(input_path, output_path, profile=fast)

# Skip automatic setparams probing
transcode_video(input_path, output_path, auto_fix_colorspace=False)

# Legacy h264-in-AVI sources: re-stamp timestamps so ffmpeg keeps every frame.
# The rate is measured from an MP4, MOV or AVI; pass frame_rate= for anything else.
transcode_video(input_path, output_path, normalize_cfr=True)
transcode_video(input_path, output_path, normalize_cfr=True, frame_rate=Fraction(500))

# Hold the decoder, filters and each encoder to 8 threads, e.g. in an 8-CPU job
transcode_video(input_path, output_path, threads=8)

transcode_video() raises RuntimeError when the output holds a different number of frames than ffmpeg decoded from the source; pass fail_on_frame_drop=False to accept such an output.

Preview and Poster Derivatives

transcode_video() can also write a browser-playable preview and a JPEG poster, as extra outputs of the same ffmpeg process, under the names the standard gives them:

from aind_video_utils import transcode_video

transcode_video(
    input_path,
    camera_dir / "video.mp4",                  # archival, frame-exact
    preview_fps=30.0,                          # preview.mp4: every N-th source frame
    metadata_csv=camera_dir / "metadata.csv",  # preview_metadata.parquet: ReferenceTime of each preview frame
    poster=True,                               # poster.jpg: sRGB still of the middle frame
)

The poster's middle frame comes from the frame count the container records. Matroska records none, so for an .mkv it is estimated from the duration; count_frames=True counts packets for the exact frame instead, at the cost of reading the whole source once more, which for a URL is a second download. A recording killed mid-write records no duration either, and its packets are counted regardless. If a truncated file's duration places the poster past its end, ffmpeg fails the run, though the archival video it wrote is complete.

The standard requires preview_metadata.parquet beside every preview. Its row k is row kN of metadata.csv, so metadata.csv must have one row per frame of the archival video; transcode_video() raises otherwise and writes no sidecar. It needs the parquet extra. write_preview_metadata() writes one on its own. The values are copied as recorded, flaws included. When CameraFrameNumber skips or steps backwards, the times may need repair (see Acquisition Timing), which needs the rows the preview drops, so a PreviewTimingWarning says to repair from metadata.csv itself.

The derivatives take these names when the archival video is named video.<ext>, as the standard names it. Otherwise they take its stem, so BottomCamera.mp4 gets BottomCamera_preview.mp4, BottomCamera_preview_metadata.parquet and BottomCamera_poster.jpg, and two videos in one folder keep theirs apart.

N puts the preview between 25 and 35 fps, preferring a whole-number rate:

Source N Preview
1000 fps 40 25 fps
500 fps 20 25 fps
300 fps 10 30 fps
240 fps 8 30 fps
120 fps 4 30 fps
29.97 fps 1 29.97 fps

The poster branches off the source ahead of the BT.709 chain, because browsers read JPEG as sRGB. The behavior video standard gives the rationale for both.

To build the command yourself, attach derivatives to a conditioned profile. Without with_setparams, the poster's zscale has no transfer function to resolve, and its failure takes the archival encode down with it. The poster's middle frame comes from the probe's frame count, or its duration when it has none; probe(input_path, count_packets=True) gives an .mkv an exact count. Pass filename=None to with_preview or with_poster to name a derivative from the primary's stem and a suffix instead, as in clip_preview.mp4:

from aind_video_utils import (
    OFFLINE_8BIT, probe, with_poster, with_preview, with_setparams,
)

probe_json = probe(input_path)
conditioned = with_setparams(OFFLINE_8BIT, probe_json)
profile = with_poster(with_preview(conditioned, probe_json), probe_json)

profile.output_paths(output_path)  # [video.mp4, preview.mp4, poster.jpg]
profile.ffmpeg_graph_args()        # ["-filter_complex", "[0:v]setparams=...,split=2[chain][d1];..."]
profile.ffmpeg_output_groups()     # one argument list per output, in the same order

ffmpeg_output_args() raises for a profile with derivatives, since one -vf cannot branch. The aind-transcode CLI does not expose derivatives yet.

Transcode CLI

With the transcode extra installed, the aind-transcode command is available:

aind-transcode videos/                                # defaults: offline-8bit, auto-fix
aind-transcode videos/ --profile offline-10bit        # explicit profile
aind-transcode videos/ --preset veryfast              # override speed
aind-transcode videos/ --crf 20 --preset veryfast     # override quality + speed
aind-transcode videos/ --no_auto_fix_colorspace true  # skip setparams probing
aind-transcode videos/ --normalize_cfr true           # legacy h264-in-AVI sources
aind-transcode videos/ --normalize_cfr true --frame_rate 500  # a container without a measurable rate
aind-transcode videos/ --jobs 4                       # parallel workers

Settings can also be stored in aind-transcode.toml in the working directory:

input = ["videos/"]
output_dir = "transcoded"
profile = "offline-8bit"
overwrite = false

MP4 Timelines

An MP4 stitched from separately encoded chunks with concat -c copy can carry notches: every frame after a seam is stamped a few frame periods late, so a reader that seeks to frame k at k / fps, as NVIDIA DALI does, lands on the wrong frame. read_mp4_frame_index() reads the sample tables from the moov alone, locally or over HTTP, and timeline_notches() reports where the timestamps step by anything other than one frame period. restamp_uniform() removes the notches without re-encoding and checks the result before returning it. transcode_video() warns with NotchedTimelineWarning when an MP4 or MOV source is notched, since the output keeps the notches unless normalize_cfr=True.

from pathlib import Path

from aind_video_utils import read_mp4_frame_index, restamp_uniform

notches = read_mp4_frame_index("bottom_camera.mp4").timeline_notches()
if not notches.is_uniform:
    restamp_uniform("bottom_camera.mp4", Path("/scratch/bottom_camera.mp4"))

The aind-mp4-timeline command needs no extra:

aind-mp4-timeline notches a.mp4 https://host/b.mp4 s3://aind-open-data/.../c.mp4 -o notches.csv
aind-mp4-timeline notches --list videos.txt -j 16      # one source per line
aind-mp4-timeline restamp bottom_camera.mp4 /scratch/bottom_camera.mp4

Acquisition Timing

Each camera's per-frame CSV, the file standard's metadata.csv or the legacy headerless <camera>.csv, reads into one table. repair_frame_times() corrects what the recording got wrong and names each repair it applies, so the caller can record it: a trigger time out of order with both its neighbours, frames paired with triggers by arrival so that each dropped frame shifts every later time, and a frame counter that steps backwards and so counts no exposures.

from aind_video_utils import read_frame_timing, repair_frame_times

table = read_frame_timing(Path("bottom_camera.csv"))
times = repair_frame_times(table.reference_time, table.frame_number)
times.time, times.dropped_before, times.repairs  # e.g. ("zip_pairing_repaired",)

Whether frames were paired by arrival is detected from the times at the counter's skips; pass pairing="by_arrival" or "by_exposure" to state it, as you must when the skips fit neither.

QC CLI

With the plotting extra installed, the aind-video-qc command is available:

# Compare linear-light input against BT.709-encoded output
aind-video-qc linear-to-bt709 input.mp4 output.mp4

# Compare ffmpeg luma extraction with OpenCV decode
aind-video-qc opencv input.mp4

# Options
aind-video-qc linear-to-bt709 input.mp4 output.mp4 --frame-time 1.5 --dpi 300 -o qc.png

QC Python API

from aind_video_utils.video_qc import compare_linear_to_bt709

fig = compare_linear_to_bt709("input.mp4", "output.mp4", frame_time=0)
fig.savefig("qc.png")

Contributing

See CONTRIBUTING.md for setup, checks and conventions.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Metadata

Release files for aind-video-utils 0.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aind-video-utils 0.10.0
File Size Uploaded Provenance Actions
aind_video_utils-0.10.0.tar.gz 257.5 kB Without provenance Details for aind_video_utils-0.10.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aind-video-utils 0.10.0
File Interpreter ABI Platform Actions
aind_video_utils-0.10.0-py3-none-any.whl Python 3 none any Details for aind_video_utils-0.10.0-py3-none-any.whl

Total release size: 344.0 kB

Release files / aind_video_utils-0.10.0.tar.gz

Download URL aind_video_utils-0.10.0.tar.gz
Size 257.5 kB
Tags Source
SHA-256 checksum
How to use checksums
b222c3e761ce92df3a2f12d38a924c9a65515d7db2dd1bf352107070489307ef
BLAKE2b-256 checksum
How to use checksums
229b349283552b08a91e665a8d20373efcc82db2ae3d7b18e2b945ae5122b667
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / aind_video_utils-0.10.0-py3-none-any.whl

Download URL aind_video_utils-0.10.0-py3-none-any.whl
Size 86.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e45c3db78ede028f2eceb6ddacfc7a0971790cc642d337ac85458313bebc9939
BLAKE2b-256 checksum
How to use checksums
d54b448e1a4800d43450c9df6bfa9e656193b20fe9a62ab02111e6b3f5db15f4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for aind-video-utils 0.10.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page