Montaj

Montaj

A video editing CLIP for AI agents. CLI-first, agent-native, open source.

Montaj is a CLIP — a CLI Program for agents. It clips onto your existing AI agent (Claude Code, OpenClaw, Cursor, or any harness) and gives it the specialized tools to edit video. Built-in steps cover the full editing pipeline. The agent decides what to run, in what order, and with what parameters.

The fundamental dependency is an agent. Montaj doesn't edit on its own. It provides the tools; the agent makes the creative decisions.

Quick install

Send this to your agent:

Install Montaj from https://github.com/theSamPadilla/montaj, then read skills/onboarding/SKILL.md to get us started.

Install

PyPI:

pip install montaj
# install Node.js >=18 separately: https://nodejs.org
montaj install whisper   # whisper-cpp binary + model weights
montaj install ui        # npm deps + UI build

Homebrew (macOS) — installs Node.js and all Python deps:

brew install theSamPadilla/montaj/montaj
montaj install ffmpeg   # pinned static ffmpeg/ffprobe (with zscale for HDR); also in `install all`

From source:

git clone https://github.com/theSamPadilla/montaj
cd montaj
pip install -e ".[connectors]"
montaj install whisper
montaj install ui

Optional extras:

pip install "montaj[connectors]"  # Kling, Gemini, OpenAI API connectors
pip install "montaj[rvm]"         # background removal (torch + RVM)
pip install "montaj[demucs]"      # audio stem separation
montaj install all                # whisper + ui + rvm

Quick Start

# Headless — agent edits, renders, done
montaj run ./clips --prompt "tight cuts, remove filler, 9:16"

# With UI — watch the agent work live, tweak the result
montaj serve

# AI video generation — agent creates from a text prompt via Kling
montaj serve   # then create an ai_video project in the UI

What's Inside

steps/              Step executables + JSON schemas (probe, trim, transcribe, generate, etc.)
workflows/          Editing plans (clean_cut, overlays, ai_video, lyrics_video, etc.)
skills/             Agent skill contracts (onboarding, edit-session, ai-video-plan, ai-video-generate, etc.)
connectors/         API connectors (Kling, Gemini, OpenAI)

render/             React + Puppeteer + ffmpeg render engine
serve/              Local HTTP + SSE server (montaj serve)
ui/                 Browser UI (Vite + React + Tailwind)
docs/               Architecture, CLI reference, UI design, schemas

How It Works

1. Upload clips + write an editing prompt (or describe an AI video)
2. montaj creates project.json [pending]
3. Agent picks it up, reads the workflow, calls steps as tools
4. Agent writes project.json as it works → UI updates live via SSE
5. Agent marks project [draft]
6. Human reviews in browser (optional) → tweaks → marks [final]
7. Render engine → final MP4

CLI

Every operation is available from the terminal. montaj is the full command, mtj is the short alias.

montaj run ./clips --prompt "tight cuts, upbeat pacing"
montaj serve
montaj render
montaj fetch https://youtube.com/watch?v=...

See the CLI Reference for the full documentation.

Steps & Workflows

Steps are individual editing operations — Python executables with JSON schemas, callable as agent tools, CLI commands, or API calls. Native steps ship with Montaj; custom steps are any executable that follows the output convention.

Category Steps
Inspect probe, snapshot, analyze_media
Clean waveform_trim, rm_fillers, rm_nonspeech
Edit materialize_cut, resize, extract_audio, crop_spec
Enrich transcribe, caption, normalize (full-source, eager default), normalize_window (per clip-window, lazy), lyrics_sync, lyrics_render
Generate kling_generate, generate_image, eval_scene
VFX remove_bg, stem_separation
Acquire fetch — download from any URL via yt-dlp

Workflows are suggested editing plans — which steps to use and with what default params. The agent reads the plan, reads the prompt, and decides the actual execution.

Workflow Description
clean_cut Trim, remove filler, clean audio
overlays Add animated overlays and titles
ai_video Generate video from text via Kling + storyboard
lyrics_video Sync lyrics to audio with animated captions
animations Custom JSX animation compositions
explainer Educational/explainer video style
floating_head Speaker overlay on background footage
clips Long-form horizontal source → series of short vertical clips with overlays; imports lazily (settings.normalize: "lazy") so each clip-window is normalized on demand rather than re-encoding the entire source

Custom steps and workflows are discovered automatically — no registration needed. See the Steps Reference and Core Concepts for details.

Skills

Skills are agent-readable contracts that teach the agent how to approach a specific editing task. Each skill describes the goal, the steps to use, the parameter choices, and the quality criteria.

Available skills: onboarding, edit-session, ai-video-plan, ai-video-generate, eval-scenes, overlay, write-overlay, animation-sections, lyrics-video, style-profile, serve, parallel, select-takes, waveform-silence, camera-vocabulary, workflow-builder, mcp.

Connectors

API connectors for external services. Installed via pip install "montaj[connectors]".

Connector Used for
Kling AI video generation (v3-omni, video-o1)
Gemini Media analysis, scene evaluation, image generation
OpenAI Image generation, analysis

Render Engine

React + Puppeteer + ffmpeg. Reads project.json [final], renders captions and overlays frame-by-frame via headless Chrome, composites with source footage via ffmpeg → final MP4.

See the Render Engine docs for the full breakdown.

UI

Optional browser interface. Upload → watch the agent work live → review → render.

montaj serve   # http://localhost:3000
  • Editor — timeline, preview player, caption editor, overlay editor
  • Storyboard — AI video scene planning with image/style references
  • Workflows — n8n-style node graph for building editing plans
  • Overlays — live animated preview of custom JSX overlays
  • Profiles — view creator style profiles (pacing, color palette, editorial direction)

The UI is a layer on top of the CLI, not a separate system. Every action maps to a CLI command.

Project JSON

The single format that flows through the entire pipeline. One file, three states:

State Who writes Contents
pending montaj serve / montaj run Clip paths, prompt, workflow name
draft Agent Trim points, ordering, captions, overlays — complete edit
final Human (via UI) Reviewed and tweaked, ready to render

For AI video projects, the storyboard (scenes, image refs, style refs) lives inside the same project.json.

See the Core Concepts page and docs/schemas/project.md for the full schema.

Docs

Full documentation at docs.montaj.ag:

Internal references (for contributors):

License

MIT — do whatever you want. See LICENSE.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (4.2.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for thesampadilla from gravatar.com thesampadilla

Credits

Author: Sam Padilla

License expression

MIT
View SPDX License List

Requires

Python >=3.10

Provides Extra

test rvm demucs connectors

Tags

video editing ai agent cli ffmpeg montaj

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

montaj-4.0.0.tar.gz (9.2 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

montaj-4.0.0-py3-none-any.whl (9.4 MB view details)

Uploaded Python 3

File details

Details for the file montaj-4.0.0.tar.gz.

File metadata

  • Download URL: montaj-4.0.0.tar.gz
  • Upload date:
  • Size: 9.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for montaj-4.0.0.tar.gz
Algorithm Hash digest
SHA256 719e875e828e56c2af38cf78e559337828b8f2a9f1ff2a819a213e68d69c260a
MD5 a3546ed36a1fd754dab47c7a269e6a60
BLAKE2b-256 34937503d4ede159cb0480d884e04b1d72570484f7f5adc559a01d43175d6637

See more details on using hashes here.

File details

Details for the file montaj-4.0.0-py3-none-any.whl.

File metadata

  • Download URL: montaj-4.0.0-py3-none-any.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for montaj-4.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d8c5cf3f5fc683ff39cb8eca3066221f6896523f73fbad6b7e66fe1851254f96
MD5 23f0e59308e6add3fd6e740465ca730d
BLAKE2b-256 fcf23d866408d95afd270f62f96364fc73aebe2978fe74516c027587473acb63

See more details on using hashes here.

Release history Release notifications | RSS feed

4.2.0

2 files

4.1.1

2 files

4.1.0

2 files

This release

4.0.0 This release

2 files

3.10.1

2 files

3.10.0

2 files

3.9.2

2 files

3.9.1

2 files

3.9.0

2 files

3.8.0

2 files

3.7.0

2 files

3.6.2

2 files

3.6.1

2 files

3.6.0

2 files

3.5.4

2 files

3.5.3

2 files

3.5.2

2 files

3.5.1

2 files

3.5.0

2 files

3.4.2

2 files

3.4.1

2 files

3.4.0

2 files

3.3.0

2 files

3.2.5

2 files

3.2.4

2 files

3.2.3

2 files

3.2.2

2 files

3.2.1

2 files

3.2.0

2 files

3.1.0

2 files

3.0.4

2 files

3.0.3

2 files

3.0.2

2 files

3.0.1

2 files

3.0.0

2 files

2.7.4

2 files

2.7.3

2 files

2.7.2

2 files

2.7.1

2 files

2.7.0

2 files

2.6.2

2 files

2.6.1

2 files

2.6.0

2 files

2.5.10

2 files

2.5.9

2 files

2.5.8

2 files

2.5.7

2 files

2.5.6

2 files

2.5.5

2 files

2.5.4

2 files

2.5.3

2 files

2.5.2

2 files

2.5.1

2 files

2.4.1

2 files

2.4.0

2 files

2.3.2

2 files

2.3.1

2 files

2.3.0

2 files

2.2.4

2 files

2.2.3

2 files

2.2.2

2 files

2.2.1

2 files

2.2.0

2 files

2.1.3

2 files

2.1.2

2 files

2.1.1

2 files

2.1.0

2 files

2.0.6

2 files

2.0.5

2 files

2.0.4

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page