Montaj
A video editing CLIP for AI agents. CLI-first, agent-native, open source.
Want the app? Montaj is also a free video editor for Mac, built on this engine: montaj.ag.
Montaj is a CLIP — a CLI Program for agents. It clips onto your existing AI agent (Claude Code, OpenClaw, Cursor, or any harness) and gives it the specialized tools to edit video. Built-in steps cover the full editing pipeline. The agent decides what to run, in what order, and with what parameters.
The fundamental dependency is an agent. Montaj doesn't edit on its own. It provides the tools; the agent makes the creative decisions.
Quick install
Send this to your agent:
Install Montaj from https://github.com/theSamPadilla/montaj, then read skills/onboarding/SKILL.md to get us started.
Install
PyPI:
pip install montaj
# install Node.js >=18 separately: https://nodejs.org
montaj install whisper # whisper-cpp binary + model weights
montaj install ui # npm deps + UI build
Homebrew (macOS) — installs Node.js and all Python deps:
brew install theSamPadilla/montaj/montaj
montaj install ffmpeg # pinned static ffmpeg/ffprobe (with zscale for HDR); also in `install all`
From source:
git clone https://github.com/theSamPadilla/montaj
cd montaj
pip install -e ".[connectors]"
montaj install whisper
montaj install ui
Optional extras:
pip install "montaj[connectors]" # Kling, Gemini, OpenAI API connectors
pip install "montaj[rvm]" # background removal (RVM on onnxruntime, CPU)
pip install "montaj[demucs]" # audio stem separation
montaj install all # whisper + ui + rvm
Quick Start
# Headless — agent edits, renders, done
montaj run ./clips --prompt "tight cuts, remove filler, 9:16"
# With UI — watch the agent work live, tweak the result
montaj serve
# AI video generation — agent creates from a text prompt via Kling
montaj serve # then create an ai_video project in the UI
What's Inside
steps/ Step executables + JSON schemas (probe, trim, transcribe, generate, etc.)
workflows/ Editing plans (overlays, ai_video, carousel, blank)
skills/ Agent skill contracts (onboarding, edit-session, ai-video-plan, ai-video-generate, etc.)
connectors/ API connectors (Kling, Gemini, OpenAI)
render/ React + Puppeteer + ffmpeg render engine
serve/ Local HTTP + SSE server (montaj serve)
ui/ Browser UI (Vite + React + Tailwind)
docs/ Architecture, CLI reference, UI design, schemas
How It Works
1. Upload clips + write an editing prompt (or describe an AI video)
2. montaj creates project.json [pending]
3. Agent picks it up, reads the workflow, calls steps as tools
4. Agent writes project.json as it works → UI updates live via SSE
5. Agent marks project [draft]
6. Human reviews in browser (optional) → tweaks → marks [final]
7. Render engine → final MP4
CLI
Every operation is available from the terminal. montaj is the full command, mtj is the short alias.
montaj run ./clips --prompt "tight cuts, upbeat pacing"
montaj serve
montaj render
montaj fetch https://youtube.com/watch?v=...
See the CLI Reference for the full documentation.
Steps & Workflows
Steps are individual editing operations — Python executables with JSON schemas, callable as agent tools, CLI commands, or API calls. Native steps ship with Montaj; custom steps are any executable that follows the output convention.
| Category | Steps |
|---|---|
| Inspect | probe, snapshot, analyze_media |
| Clean | waveform_trim, rm_fillers, rm_nonspeech |
| Edit | materialize_cut, resize, extract_audio, crop_spec |
| Enrich | transcribe, caption, normalize (full-source, eager default), normalize_window (per clip-window, lazy), lyrics_sync, lyrics_render |
| Generate | kling_generate, generate_image, eval_scene |
| VFX | remove_bg, stem_separation |
| Acquire | fetch — download from any URL via yt-dlp |
Workflows are suggested editing plans — which steps to use and with what default params. The agent reads the plan, reads the prompt, and decides the actual execution.
| Workflow | Description |
|---|---|
overlays |
Trim, remove filler, then add overlays (the default) |
ai_video |
Generate video from text via Kling + storyboard |
carousel |
Image carousel of still slides, rendered to PNGs |
blank |
Empty project with no steps; edit by hand |
Custom steps and workflows are discovered automatically — no registration needed. See the Steps Reference and Core Concepts for details.
Skills
Skills are agent-readable contracts that teach the agent how to approach a specific editing task. Each skill describes the goal, the steps to use, the parameter choices, and the quality criteria.
Available skills: onboarding, edit-session, ai-video-plan, ai-video-generate, eval-scenes, overlay, write-overlay, serve, parallel, select-takes, waveform-silence, camera-vocabulary, mcp.
Connectors
API connectors for external services. Installed via pip install "montaj[connectors]".
| Connector | Used for |
|---|---|
| Kling | AI video generation (v3-omni, video-o1) |
| Gemini | Media analysis, scene evaluation, image generation |
| OpenAI | Image generation, analysis |
Render Engine
React + Puppeteer + ffmpeg. Reads project.json [final], renders captions and overlays frame-by-frame via headless Chrome, composites with source footage via ffmpeg → final MP4.
See the Render Engine docs for the full breakdown.
UI
Optional browser interface. Upload → watch the agent work live → review → render.
montaj serve # http://localhost:3000
- Editor — timeline, preview player, caption editor, overlay editor
- Storyboard — AI video scene planning with image/style references
- Workflows — n8n-style node graph for building editing plans
- Overlays — live animated preview of custom JSX overlays
- Profiles — view creator style profiles (pacing, color palette, editorial direction)
The UI is a layer on top of the CLI, not a separate system. Every action maps to a CLI command.
Project JSON
The single format that flows through the entire pipeline. One file, three states:
| State | Who writes | Contents |
|---|---|---|
pending |
montaj serve / montaj run |
Clip paths, prompt, workflow name |
draft |
Agent | Trim points, ordering, captions, overlays — complete edit |
final |
Human (via UI) | Reviewed and tweaked, ready to render |
For AI video projects, the storyboard (scenes, image refs, style refs) lives inside the same project.json.
See the Core Concepts page and docs/schemas/project.md for the full schema.
Docs
Full documentation at docs.montaj.ag:
- Installation — Homebrew, PyPI, source
- Quick Start — first project in 2 minutes
- CLI Reference — full command list
- Steps Reference — all 27+ built-in steps
- Core Concepts — architecture, project.json, trim specs, workflows
- Render Engine — the compositing pipeline
- Agent Integration — MCP, HTTP API, skills
- Connectors — Kling, Gemini, OpenAI setup
- Extending — custom steps, workflows, connectors
Internal references (for contributors):
- Architecture — deep implementation details
- Project JSON Schema — field-level schema reference
- Overlay Contract — render component spec
License
MIT — do whatever you want. See LICENSE.
1 maintainer
Metadata
1 maintainer
Release files for montaj 5.19.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| montaj-5.19.7.tar.gz | 9.9 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| montaj-5.19.7-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 20.1 MB
Release files / montaj-5.19.7.tar.gz
| Download URL | montaj-5.19.7.tar.gz |
|---|---|
| Size | 9.9 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7b7566947c0269c127329eca4e98203809ced0a8ad06ecfac95587483b121bd6
|
|
BLAKE2b-256 checksum How to use checksums |
07c85c695651905534eafb9013ec98602d23584bfb744138a863d875c5bf04b8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.7
|
Release files / montaj-5.19.7-py3-none-any.whl
| Download URL | montaj-5.19.7-py3-none-any.whl |
|---|---|
| Size | 10.2 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f8e28c195d5b8c7b0c2e814f2db9eae1447a1a14449b053bcd738daa73d276bc
|
|
BLAKE2b-256 checksum How to use checksums |
66455dcd3c9df1c48757032261c6eae1f0f94c91dd67cbfc5795686b36b6a26a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.7
|
