Capture Helper

🇫🇷 · 🇬🇧

Capture Helper belongs to a collection of libraries called AI Helpers developed for building Artificial Intelligence.

Local-first library that turns a live camera or microphone into data the rest of the AI Helpers suite already knows how to consume, plus a browser GUI for arranging several such sources at once. A camera feed becomes a stream of ordinary images: iter_camera_frames yields one array per frame, a grid of pixel colors with a height, a width, and three color channels stored blue, then green, then red (the format OpenCV and the rest of this suite use, called BGR uint8), matching video-helper's extract_frames. A microphone feed becomes a stream of sound samples: iter_mic_audio yields MicFrames, small chunks of raw audio measurements taken over time (the standard format is called pulse-code modulation, PCM for short), matching podcast-helper's extract_audio_stream. Because both iterators speak the same shapes as their sibling packages, code written against a recorded video file or a downloaded podcast runs unchanged against a live camera or microphone. On top of that, the GUI lets you drag several live sources onto a canvas, preview them in the browser, and save the layout as a reusable JSON scene the CLI or API can replay later. The two iterators are the stable foundation; the scene configurator, added more recently, builds on top of them.

🌍 AI Helpers

The Promise

Local-first by design. capture-helper runs entirely on your machine; camera and microphone data is captured and processed locally, never uploaded to any third-party service, with no telemetry, no account, no cloud lock-in. Part of the AI Helpers suite: sovereignty over your data through local-first Open Source.

Battle-tested, not a prototype. Every change goes through the same gate before it ships: the full test suite and a lint pass run in continuous integration (see the CI badge above). Only a green run gets tagged and published to PyPI under semantic versioning. Downstream, vocal-helper depends on capture-helper's live microphone path to feed its speech pipeline in production, so the camera and microphone iterators carry real workloads, not just their own tests.

Documentation

💻 Documentation

🗺️ Landscape

📋 Examples

Features

Here is exactly what exists today.

Capture layer (stable contracts)

• SourceKind literal ("camera" | "microphone")
• Source typed dict (kind, name, index, platform, driver)
• MicFrame typed dict, mirroring podcast_helper.PcmFrame
• list_sources(kind=None): cross-platform device enumeration via ffmpeg -list_devices (macOS avfoundation, Windows dshow, Linux v4l2 + pulse)
• pick_source(kind, *, name_substring=..., index=...): picks the first matching device, raises ValueError if nothing matches
• iter_camera_frames(source, *, width=..., height=..., output_width=..., output_height=..., fps=..., max_frames=...): yields (H, W, 3) BGR uint8 numpy arrays, the same contract as video_helper.extract_frames
• iter_mic_audio(source, *, target_sample_rate=16000, to_mono=True, frame_ms=20): async iterator yielding MicFrames, the same contract as podcast_helper.extract_audio_stream
• ffmpeg_input_args(source): a low-level helper exposed for users wiring their own ffmpeg pipelines

Live multi-source scene configurator (new, additive)

• Browser GUI at GET /gui: enumerate all cameras and microphones, drop them onto a 16:9 canvas, live-preview each camera as an in-browser MJPEG stream, watch live microphone level meters, drag them into place, then save the visual design as a reusable JSON scene (and load one back). No build step: vanilla JS + Tailwind CDN.
• Scene model: Scene / SceneSource typed dicts, new_scene(...), add_source(...), validate_scene(...), save_scene(...), load_scene(...), resolve_scene_sources(...) (maps a scene onto the current machine's devices), scene_from_available_devices(...).
• Live-preview primitives: snapshot_jpeg(source) (one live JPEG), iter_camera_jpeg(source) (JPEG stream for MJPEG), mic_level(source) (RMS / peak dBFS for a VU meter), frame_to_jpeg(frame).
• Scene CLI: capture-helper scene-auto (auto-populate from devices), scene-validate, scene-show (reports how each source resolves here).
• Scene / preview HTTP endpoints: GET /scene, POST /scene/save, POST /scene/load, GET /preview/camera.jpg, GET /preview/camera.mjpeg, GET /preview/mic-level.

What makes this more than a live-preview widget is that the layout itself becomes data: arranging cameras and microphones on the canvas produces a portable JSON file, so the same scene can be replayed later, on a different machine, from the command line, with no browser involved.

import asyncio
import capture_helper as ch

# Enumerate available devices
for s in ch.list_sources():
    print(f"{s['kind']:10s} [{s['index']}] {s['name']:40s} (driver={s['driver']})")
    # camera     [0] FaceTime HD Camera                       (driver=avfoundation)
    # microphone [0] Built-in Microphone                      (driver=avfoundation)

# Camera → numpy BGR frames (drop-in for video_helper.extract_frames)
cam = ch.pick_source("camera")
for frame in ch.iter_camera_frames(cam, output_width=640, output_height=360,
                                   fps=30, max_frames=300):
    # frame.shape == (360, 640, 3), dtype uint8, BGR.
    do_something(frame)

# Microphone → async PCM stream (drop-in for podcast_helper.extract_audio_stream)
async def listen():
    mic = ch.pick_source("microphone")
    async for f in ch.iter_mic_audio(mic, target_sample_rate=16000,
                                     to_mono=True, frame_ms=20):
        # f["pcm"].shape == (320,), 20ms of audio at 16kHz mono.
        await asr.feed(f["pcm"])
asyncio.run(listen())

Installation

Prerequisites: Python 3.10-3.13 and git, ffmpeg, PortAudio, cross-platform:

• 🍎 macOS (Homebrew): brew install python git ffmpeg portaudio
• 🐧 Ubuntu/Debian: sudo apt update && sudo apt install -y python3 python3-pip git ffmpeg portaudio19-dev
• 🪟 Windows (PowerShell): winget install Python.Python.3.12 Git.Git Gyan.FFmpeg (PortAudio ships inside the Python wheels)

We recommend using Python environments. Check this link if you're unfamiliar with setting one up: 🥸 Tech tips.

You still need ffmpeg on PATH for device enumeration and live capture to return anything.

From PyPI (recommended)

# Core INPUT layer (list/pick sources, camera + mic iterators)
pip install capture-helper

# Optional surfaces
pip install "capture-helper[cli]"       # click-based CLI twin
pip install "capture-helper[api]"       # FastAPI HTTP surface

From source (no PyPI)

git clone https://github.com/warith-harchaoui/capture-helper.git
cd capture-helper
pip install -e .

# Optional surfaces
pip install -e ".[cli]"
pip install -e ".[api]"

Roadmap

For a full cookbook (per-OS ffmpeg input strings, snapshot capture, live preview, scene save/load, ASR / VAD wiring), see 📋 EXAMPLES.md. For the exhaustive trigger catalogue, see 📋 TRIGGERS.md.

Multi-surface exposure

capture-helper ships the same capabilities through six surfaces, so it plugs in wherever you already work with no rewrite needed.

# CLI (argparse, always available)
capture-helper list-sources
capture-helper pick-source --kind camera --name FaceTime
capture-helper capture-mic --output mic.wav --seconds 3
capture-helper stream-mic | my-live-consumer   # raw f32le PCM, unbounded

# CLI (click twin, same subcommands)
capture-helper-click list-sources
capture-helper-click capture-camera --output-dir frames/ \
    --output-width 640 --output-height 360 --max-frames 30

# HTTP surface
uvicorn capture_helper.api:app --host 0.0.0.0 --port 8000
curl http://localhost:8000/sources
curl -o frames.zip \
    'http://localhost:8000/capture/camera?output_width=320&output_height=240&max_frames=10'

# Browser GUI: live multi-source scene configurator
uvicorn capture_helper.api:app --port 8000
# open http://localhost:8000/gui  (or just http://localhost:8000/)

# Docker (ships FastAPI + GUI by default)
docker build -t capture-helper .
docker run --rm -p 8000:8000 capture-helper

The GUI at /gui is the live multi-source scene configurator: it enumerates your cameras and microphones, live-previews each camera (MJPEG) and each mic (level meter), lets you arrange them on a canvas, and saves the design as a reusable .scene.json the CLI / API can replay. See 📋 GUI.md.

Rust rewrite

A companion crate, capture-helper-rs, rewrites the microphone half — device enumeration plus a streaming MicFrame iterator on top of cpal — in native Rust. No camera, no GUI, no video, just the audio-capture core. The interest: it talks to CoreAudio/WASAPI/ALSA-PulseAudio-JACK-PipeWire directly, so it needs neither ffmpeg nor a Python runtime on the target machine — a better fit when live mic capture is embedded inside a Rust application (a native app, an embedded device) rather than orchestrated from Python.

Author

• Warith HARCHAOUI

Acknowledgements

Special thanks to Mohamed Chelali and Bachir Zerroug for fruitful discussions.

License

This project is licensed under the BSD-3-Clause License; see the LICENSE file for details.