Video Helper

🇫🇷 · 🇬🇧

Video Helper belongs to a collection of libraries called AI Helpers developed for building Artificial Intelligence.

🌍 AI Helpers

Video Helper is a Python library that provides utility functions for processing video files. It includes features like loading, converting, extracting frames as well as working with subtitle formats.

The Promise

Local-first by design. video-helper runs entirely on your machine. Everything is processed locally with open-source tooling (ffmpeg): your data is never uploaded to a third-party service, no telemetry, no account, no cloud lock-in. You own the whole pipeline. Part of the AI Helpers suite: sovereignty over your data through local-first Open Source.

Battle-tested

video-helper follows the same discipline as the rest of the suite. Every change runs through continuous integration before it reaches main; CI stays green, never red on main. Each release is tagged with semantic versioning (currently v2.3.2); that exact tag is what's published on PyPI, so what you pip install matches what was tested. It builds on os-helper, the suite's shared foundation for logging and file handling. youtube-helper, the suite's download layer, depends on video-helper directly for every clip it hands off for processing. This is meant to be built on, not experimented with.

Documentation

💻 Documentation

🗺️ Landscape

📋 Examples

Features

• Video validation: is_valid_video_file, extension check plus an ffmpeg.probe round-trip.
• Conversion: video_converter, re-encode, resample fps, resize (aspect-preserving), strip audio.
• Frame access: extract_frames (generator with time/index range, stabilization, sampling) and dump_frames (list → video).
• Optical flow: a per-pixel estimate of motion between two frames (vx/vy, how far each pixel shifted sideways and vertically). iter_frame_optical_flow wraps any BGR frame iterator with dense vx/vy, color or grayscale=True (DIS/Farneback free, RAFT via the [flow] extra), extract_optical_flow is the video-file convenience wrapper (.mp4 visualization or raw .npy), and resize_flow is a wavelet-based, discontinuity-preserving flow resize.
• Temporal crop: extract_video_chunk, video_duration.
• Pipeline primitives: black_video, compress_video, image_loop_to_video, concat_videos, overlay_image, extract_audio_track, mux_audio_video, burn_subtitles.
• Subtitles: srt2vtt (with companion CSS), extract_unique_colors.
• Face-anchored speaker identity (video_helper.faces, needs the [faces] extra): audio-only diarization tells you a voice cluster exists but not which on-screen face it belongs to. This submodule answers that by detecting faces (YuNet), tracking them across frames, and scoring which tracked face's lip motion lines up with a given speaker's audio activity, a technique called active-speaker detection (ASD: catching who is actually talking on screen, not just whose voice is on the track). FaceDetector / FaceRecognizer (YuNet + SFace, OpenCV's own DNN wrappers, no HuggingFace at runtime), track_faces (IoU tracking), get_engine (a zero-weight lip-motion proxy, or the accurate Light-ASD PyTorch model), and active_speaker_map, the harness that ties it together: it samples a handful of short clips instead of decoding the whole recording, growing the sample only for speakers it isn't yet sure about. See the faces module docstring for the full picture.

Installation

Prerequisites: Python 3.10–3.13 and git, ffmpeg, cross-platform:

• 🍎 macOS (Homebrew): brew install python git ffmpeg
• 🐧 Ubuntu/Debian: sudo apt update && sudo apt install -y python3 python3-pip git ffmpeg
• 🪟 Windows (PowerShell): winget install Python.Python.3.12 Git.Git Gyan.FFmpeg

We recommend using Python environments. Check this link if you're unfamiliar with setting one up: 🥸 Tech tips.

From PyPI (recommended)

# Core video utilities (library + argparse CLI)
pip install video-helper

# Optional surfaces and backends
pip install "video-helper[pyav]"      # PyAV frame backend
pip install "video-helper[cli]"       # click-based CLI twin
pip install "video-helper[api]"       # FastAPI HTTP surface
pip install "video-helper[faces]"     # face-anchored speaker identity (YuNet / SFace / Light-ASD)

From source (no PyPI)

git clone https://github.com/warith-harchaoui/video-helper.git
cd video-helper
pip install -e .

# Optional surfaces and backends
pip install -e ".[pyav]"
pip install -e ".[cli]"
pip install -e ".[api]"

Usage

For the full catalog of recipes, see 📋 EXAMPLES.md.

Here’s an example of how to use Video Helper to load, convert, and extract frames from a video file:

import video_helper as vh

# Check if the video file is valid
video_file = "example.mp4"
valid = vh.is_valid_video_file(video_file)  # True or False

# Get video dimensions and details
details = vh.video_dimensions(video_file)
print(details)
# {'width': 1920, 'height': 1080, 'duration': 10.0, 'frame_rate': 30.0, 'has_sound': True}

# Convert the video file to a different format
output_video = "video_tests/example_converted.mp4"
vh.video_converter(video_file, output_video, frame_rate=30, width=640, without_sound=True)

# The images will never be distorted:
# aspect ratios are kept even for arbitrary width and height thanks to black padding if necessary

# Extract frames from the video

start_instant = 5  # seconds
# it corresponds to start_index = start_instant * frame_rate = 5 * 30 = 150th frame

end_instant = 10  # seconds
# it corresponds to end_index = end_instant * frame_rate = 10 * 30 = 300th frame

frame_step = 5  # take one frame every 5
# which corresponds to 1 frame every 5 / frame_rate = 5 / 30 = 0.17 second

# This means that in the video we take 1 frame every 5 from the 150th to the 300th

# List example
frames = list(
    vh.extract_frames(
        video_file, start_instant=start_instant, end_instant=end_instant, frame_step=frame_step
    )
)

# For loop example
for frame in vh.extract_frames(
    video_file, start_instant=start_instant, end_instant=end_instant, frame_step=frame_step
):
    pass  # Replace with your frame processing logic

# Each frame is a numpy array with shape (height, width, channels)
# with pixel values between 0 and 255.

Another example is about subtitles

Convert SRT subtitles to WebVTT with color preservation:

import video_helper as vh

srt_file = "subtitles.srt"
vtt_file = "subtitles.vtt"
css_file = "subtitles.css"

vh.srt2vtt(srt_file, vtt_file, css_file)

Multi-surface exposure

Every public function is reachable from five surfaces, all systematically wired (nothing is CLI-only or library-only):

The FastAPI app also serves a minimal browser GUI ("video bench") at GET /gui (and GET / redirects there): drop a clip, pick one operation, run it against the same HTTP endpoints, preview input vs output in an in-browser <video> / <img> player, and download the result. It is a single self-contained page (Tailwind via CDN + vanilla JS, no build step) defined in video_helper/gui.py.

pip install 'video-helper[api]'
uvicorn video_helper.api:app --port 8000
# open http://localhost:8000/gui  (or just http://localhost:8000/)

The Dockerfile at the repo root ships .[api,pyav] by default on python:3.11-slim with ffmpeg and libass: one docker build && docker run -p 8000:8000 gives you the HTTP + GUI surfaces immediately.

For the exhaustive catalogue of what triggers each operation (natural-language phrasings, commands, functions, file types), see TRIGGERS.md.

See GUI.md for the roadmap toward a richer GUI (Recipe Canvas, frame-first comparator, batch drop zone: the minimal /gui bench above is the first step).

Use it as an agent skill

skills/video-helper/ packages video-helper as a Claude Skill and an OpenCode skill, so an agent can reach for it on your behalf. See skills/README.md to install it (symlink into ~/.claude/skills/ and ~/.opencode/skills/), and TRIGGERS.md for the exhaustive trigger catalogue — scripts/check_triggers.py keeps the skill's description in sync with it, and the test suite fails if a capability stops being named.

API Reference

By default frames are BGR numpy.ndarray of shape (H, W, 3) with pixel values in [0, 255]. See EXAMPLES.md → Destination for the full shape × colorspace table including torch (CHW/NCHW/CTHW RGB) and PIL (RGB, size=(W, H)).

Author

• Warith HARCHAOUI

Acknowledgements

Special thanks to Mohamed Chelali and Bachir Zerroug for fruitful discussions.

License

This project is licensed under the BSD-3-Clause License: see the LICENSE file for details.