premiere-ai

AI-assisted video-production workflows for Adobe Premiere Pro, built on top of premiere-cli (which provides the premiere-cli/premiere-log CLIs and the Premiere Bridge CEP panel).

Installation

pip install -e .

Commands

transcribe <input_file>

Transcribes an audio or video file in two passes:

1. VibeVoice-ASR (mlx-community/VibeVoice-ASR-4bit) — produces a transcript in .txt, .srt, or .vtt format
2. Qwen3-ForcedAligner (mlx-community/Qwen3-ForcedAligner-0.6B-4bit) — aligns each ASR segment to produce word-level timestamps saved as .words.json

Video files have their audio extracted automatically via ffmpeg before transcription.

transcribe video.mp4
transcribe audio.wav --format srt --language English
transcribe audio.m4a --output transcript --verbose

Options:

Output files:

• <stem>.txt / .srt / .vtt — transcript
• <stem>.words.json — word-level timestamps as [{"text": "word", "start": 0.123, "end": 0.456}, …]

Prerequisites — download models before first use:

hf download mlx-community/VibeVoice-ASR-4bit
hf download mlx-community/Qwen3-ForcedAligner-0.6B-4bit

remove-pauses <input_file>

Detects pauses to remove from an audio or video file:

1. Transcribes it (reusing transcribe's output if a matching .words.json already exists next to the input).
2. Asks Claude (claude-opus-4-8, via the Anthropic API) which word-gaps are phrase/sentence boundaries — by default, only these are considered candidates for cutting. Pass --allow-mid-phrase-cuts to skip this check entirely and consider every word-gap a candidate.
3. Runs Silero VAD (via torch.hub) to confirm which candidate gaps are actually silent.
4. Narrows each confirmed pause with asymmetric frame margins controlled by --aggressiveness: a small fixed buffer after the preceding word, and a larger buffer before the next word that shrinks as aggressiveness increases (since cutting too close to the next word risks clipping it).

Each cut range is a half-open [start, end) interval: start is the first frame to delete, end is the first frame that remains — matching the standard NLE in/out-point convention.

remove-pauses recording.wav
remove-pauses video.mp4 --aggressiveness 0.8 --min-pause 250
remove-pauses interview.mp4 --allow-mid-phrase-cuts

Options:

Output:

• Printed to stdout and written to <stem>.cuts.txt: one MM:SS:FF - MM:SS:FF pause range per line, plus a final "Total pause time removed: M:SS.s" line (when any cuts are found).

Prerequisites:

• ANTHROPIC_API_KEY must be set (in the environment or a .env file) unless --allow-mid-phrase-cuts is used, since that flag skips the Claude API call entirely.
• First run downloads the Silero VAD model via torch.hub (requires network access).

zmbv-to-h265-vga <input_file> [output_file]

Converts a DOSBox ZMBV screen recording (VGA palette) to H.265/HEVC. Scales the frame up 2× using nearest-neighbour to preserve pixel-art crispness.

zmbv-to-h265-ega <input_file> [output_file]

Same as above for EGA palette recordings.

import-raw-footage <project_dir>

Locates the latest matching raw camera recording and mic recording, matches them by capture time (falling back to media duration), and copies both into <project_dir>/assets/video/ and <project_dir>/assets/audio/.

import-raw-footage /path/to/project
import-raw-footage /path/to/project --camera-file cam.mp4 --mic-file mic.wav

This package makes no assumption about your camera or mic hardware — auto-detection is entirely driven by environment variables, each optional and comma-separated for multiple locations:

Any variable left unset just means that source is skipped; passing both --camera-file and --mic-file explicitly needs none of them set. Full flag reference: import-raw-footage --help.

check-sequence-sync

Verifies that a Premiere Pro sequence's paired video and audio clips are still in sync after timeline surgery (silence removal, ripple deletes, retimes, manual in-point repairs), and that no clip re-plays source media its neighbour already played. Reads the sequence through premiere-cli get-full-sequence-info, so the bridge panel must be running.

check-sequence-sync                                   # active sequence, V1/A1
check-sequence-sync --sequence-name "final cut" --offset -1.848
check-sequence-sync --from-json backup.json --json    # a saved dump, machine-readable

Two checks, over every clip on the video track and its paired audio track (paired by identical timeline start):

• Sync offset invariant — audio.inPoint − video.inPoint must equal the sync offset for every pair. Pass the offset sync-audio reported with --offset; otherwise it is inferred per media file as the modal value, which is right whenever most pairs are still in sync.
• Source overlap — consecutive clips from the same media must not overlap in source time; when they do, the tail of one clip repeats at the head of the next.

Duration, clip count and coverage cannot detect either — a slipped clip keeps its timeline start, end and duration — which is why these checks exist. When pairs are desynced, the report names, per media file, which track still lines up without overlapping (the one to keep) and prints the in-point that slips the other track back, ready for premiere-cli trim-clip --in-point-seconds. Exit status is 0 on PASS, 1 on any finding. A clip's source out-point is always derived as inPoint + duration; the outPointSeconds Premiere reports goes stale after edits.

Full flag reference: check-sequence-sync --help.

calibrate-lut <image>

Builds a 3D .cube correction LUT from a single photo of the video page of a Calibrite ColorChecker Passport Video 2 chart: detects the chart (SAM3), measures its patches, and fits

• per-channel (R, G, B) tone curves for exposure/white-balance, from the left-panel 3-step strip + grid columns 2 and 3 against IRE targets
• a single global hue rotation aligning the six chromatic chips (col 0) to their Rec.709 vectorscope target hues, folding in the skin-tone chips (col 1) toward the classic "-I axis" skin-tone line
• highlight/shadow anchors (col 3) folded into the same tone curves, at IRE targets you can override

calibrate-lut chart_photo.png
calibrate-lut chart_photo.png --output corrected.cube --lut-size 33
calibrate-lut chart_photo.png --highlight-shadow-ire 100,97,94,6,3,0

Options:

Prerequisites — download the segmentation model before first use:

hf download mlx-community/sam3-4bit

Caveats (see the module docstring in build_lut.py for the full reasoning): per-channel WB/exposure uses real IRE targets, and the chromatic hue targets are exact (computed analytically from Rec.709) — but the skin-tone target and default highlight/shadow IRE values are reasoned estimates, not vendor-confirmed figures, so verify visually against Premiere's own skin-tone-line toggle before trusting them as exact. This only ever measures the chart's video page — see calibrate-lut-classic below for the traditional 24-patch page.

The resulting .cube is exactly the input premiere-cli's apply-lut/desktop-set-input-lut commands consume to apply the correction to a clip's Lumetri Color effect.

calibrate-lut-classic <image>

Builds a 3D .cube correction LUT from a photo of the classic (traditional 24-patch) page of an X-Rite/Calibrite ColorChecker: locates the chart (SAM3), segments its 24 squares, matches each to its published reference sRGB value (Hungarian assignment across the whole grid, so no single ambiguous patch can steal another's slot — cross-checked by finding which single rotate/flip orientation is consistent across the most matched patches, excluding any that disagree from the fit), and fits a color-correction matrix from measured colors to those references.

calibrate-lut-classic chart_photo.png
calibrate-lut-classic chart_photo.png --output corrected.cube --lut-size 33
calibrate-lut-classic chart_photo.png --matrix-method "Finlayson 2015" --matrix-degree 2

Options:

Prerequisites — same as calibrate-lut:

hf download mlx-community/sam3-4bit

Caveats: reference values are BabelColor/Danny Pascale's published nominal sRGB measurements for the Classic chart, not the physical unit's own individually-calibrated values — good enough to verify grid orientation and fit a real hue/saturation/white-balance correction against, but this fits only a color matrix, no separate exposure/tone curve. Unlike calibrate-lut, this is a single-photo, single-page pipeline throughout — it does not attempt to combine anchors from a separate video-page shot (an earlier exploration combining both pages from two unrelated photos is preserved in video-production's scratch history as a cautionary example: that combination is only physically valid when both pages are photographed together, under the same lighting, in the same frame).

vectorscope <image> [<image> ...]

Renders a Premiere-Lumetri-style YUV vectorscope trace from one or more images: the same graticule Lumetri draws (bounding circle, the six 75%/100% primary/secondary target boxes, the skin-tone line, centre crosshair), with the trace itself a log-scaled 2D histogram of the frame's Cb/Cr so dense regions read bright like a real scope's persistence.

vectorscope frame.png -o scope.png
vectorscope before.png after.png -o compare.png --labels before after

Pass --colorchecker-patches chromatic or --colorchecker-patches skin to restrict the trace to just one category of patches on a Calibrite ColorChecker Passport Video 2's video page, instead of the whole frame — useful for judging hue accuracy against just the six chromatic (green/cyan/blue/magenta/red/yellow) or six skin-tone chips. This detects the chart with the same SAM3 pipeline calibrate-lut uses (chart location, then grid recovery to isolate the fixed chromatic/skin grid columns — see patch_mask.py).

Add --debug-mask-output path.png alongside --colorchecker-patches to also save the image with everything outside the detected patches dimmed, so you can visually confirm the chart/column was located correctly before trusting the resulting trace. When rendering more than one image, each gets its own debug file (an index is inserted before the extension).

Options:

Prerequisites — only needed with --colorchecker-patches, same as calibrate-lut:

hf download mlx-community/sam3-4bit

Diagnostics (premiere_ai.colorchecker)

calibrate-lut's supporting modules also include standalone diagnostic tools for troubleshooting chart detection/segmentation on new footage — not installed as console scripts (debug tools, not end-user commands), run via python -m:

Several of these default their --image argument to a fixture image (colorchecker.png) that isn't bundled in this package — pass --image explicitly.

The chart-location step (detect.py) is currently stubbed to a fixed simulated response rather than querying a live VLM server, so downstream work isn't blocked on one being available — see its module docstring before relying on it for a real detection.

premiere-log / premiere-cli (from the premiere-cli package)

The Premiere-driving CLIs — premiere-log (send a message to the Premiere Bridge panel's log view) and premiere-cli (execute ExtendScript-backed commands against the open project) — live in the separate premiere-cli package, installed automatically as a dependency of this one. See that repo's README and docs/COMMANDS.md for the full command reference, including premiere-cli init-project, which creates a fresh empty project from a bundled template (formerly this package's create-empty-premiere-project).

File structure

src/premiere_ai/
    transcribe.py          transcription + forced alignment CLI
    pause_cuts.py          pure pause-detection logic (timecodes, margins, VAD inversion, Claude parsing)
    remove_pauses.py       remove-pauses CLI orchestration
    scripts.py             thin Python wrappers for the shell scripts
    sync_audio.py          sync-audio / audio-offset CLI
    import_raw_footage.py  import-raw-footage CLI
    check_sequence_sync.py check-sequence-sync CLI (A/V sync + source-overlap audit of a sequence)
    zmbv_to_h265_vga.sh    VGA ZMBV → H.265 conversion
    zmbv_to_h265_ega.sh    EGA ZMBV → H.265 conversion
    font_metrics.py        font measurement utilities
    colorchecker/           ColorChecker chart -> Lumetri correction-LUT pipeline
        build_lut.py            calibrate-lut CLI (video-page production command)
        build_lut_classic.py    calibrate-lut-classic CLI (classic-page production command)
        vectorscope_render.py   vectorscope CLI (renders a YUV vectorscope, optional patch masking)
        patch_mask.py           detects the video page + builds chromatic/skin patch masks
        pages.py                per-page (classic/video) chart config
        patch_grid.py           grid recovery + colorimetric self-consistency checks
        classic_reference.py    published reference sRGB values for the classic page
        vectorscope.py          Rec.709 vectorscope geometry / hue targets
        tone_curve.py           per-channel tone-curve fitting
        lut_io.py / image_io.py .cube read/write; color-managed image loading
        detect.py               VLM chart-location query (currently stubbed)
        segment.py, subsegment.py, segment_left_target.py,
        extract_targets.py, identify_video_patches.py,
        identify_classic_patches.py, apply_cube_lut.py
                                 diagnostic tools, not installed as console scripts
tests/                     pytest suite mirroring the modules above

Development

git clone https://github.com/stefanwebb/premiere-ai
cd premiere-ai
pip install -e ".[dev]"
pytest

Scope

This package contains the AI-assisted workflow layer for video production. Everything that drives Premiere Pro itself (ExtendScript-backed commands, the CEP bridge panel) lives in the separate premiere-cli package, which this one depends on.

License

CC-BY-SA-4.0.