vidmagik-agent 0.1.1

AI-powered video editing agent that creates short-form content from any video

🎬 vidmagikAgent

An AI-powered video editing agent that automatically creates short-form content from any video.

Paste a URL → AI downloads, analyzes, detects highlights, edits, and exports vertical shorts for TikTok, YouTube Shorts & Instagram Reels.

Quick Start • How It Works • Web App • MCP Backend • Custom Effects • Docker • Testing

---

---

Disclaimer - The Quality of the output video is dependant on the model you use plus your extra instructions although a model may handle tool calling very well it may not be able to produce high quality video edits in my experience its best to use a VLLM with a large context window and good reasoning capabilities or a fine tuned model for video editing tasks doesnt have to generate video just understand the context and make good decisions.

---

📖 Overview

vidmagikAgent is a full-stack AI video editing agent that turns any video into polished, platform-ready short-form content — completely autonomously.

The project consists of three major components:

Component	What It Does
AI Shorts Creator (src/app/)	A NiceGUI web application — the main user-facing product. Paste a video URL, give optional creative instructions, and the AI agent handles everything: downloading, highlight detection, scene analysis, intelligent clip selection, vertical reframing, effects, and export.
vidmagik-mcp Backend (src/api/main.py)	An MCP (Model Context Protocol) server exposing 60+ video/audio editing tools built on MoviePy. This is the "hands" the AI agent uses to actually edit video. Supports stdio, SSE, and StreamableHTTP transports.
Custom Effects Library (src/api/custom_fx/)	9 production-ready visual effects — Matrix rain, kaleidoscope, chroma key, face-tracking auto-framing, 3D rotating cube, and more — plus an optical-flow highlight detection module that goes beyond MoviePy's built-in effects.

Key Capabilities

• 🤖 Fully Autonomous — The AI agent makes all creative decisions: which moments to pick, how to crop, what effects to apply, when to cut.
• 🌐 Any Video Source — Downloads from YouTube, TikTok, Instagram, Twitter, and 1000+ sites via yt-dlp.
• 📱 Vertical-First — Auto-framing with face detection (Haar Cascades) converts 16:9 → 9:16 while keeping subjects centered.
• 🔍 Optical-Flow Highlight Detection — Automatically finds high-motion/action moments using dense optical flow (Farneback algorithm) for intelligent clip selection.
• 🔧 Any LLM — Works with LM Studio, Gemini, OpenAI, Anthropic, Ollama, or any OpenAI-compatible API via LiteLLM. Auto-detects provider from environment variables.
• 🎨 60+ Editing Tools — Full MoviePy API + 9 custom effects + highlight detection, all exposed as MCP tools the agent can call.
• 📡 Real-Time Streaming UI — Watch the agent think, call tools, and produce results in a live chat-style activity log.
• 🔌 Flexible Transport — MCP server supports stdio (local subprocess), SSE, and StreamableHTTP for scalable remote deployments.
• 🐳 Docker Ready — Multi-service docker-compose.yml for one-command deployment.

---

🚀 Quick Start

Prerequisites

Requirement	Why
Python ≥ 3.13	Runtime
FFmpeg	Video/audio encoding & decoding (used by MoviePy)
ImageMagick	Text rendering (for TextClip)

Install

vidmagikAgent is published on PyPI:

# Install from PyPI
pip install vidmagik-agent

# Or with uv
uv pip install vidmagik-agent

This installs all dependencies and registers the vidmagik CLI command.

🖥 CLI Usage

After installing, launch the web app with a single command:

vidmagik

This starts the AI Shorts Creator web UI at http://127.0.0.1:3000.

Note: Python ≥ 3.13 is required. System dependencies (FFmpeg, ImageMagick) must be installed separately.

Environment Variables (.env)

Copy .env.example to .env and configure your LLM provider. Docker Compose reads this automatically.

cp .env.example .env

The system auto-detects your LLM provider from environment variables:

Variable	Example	Notes
LM_STUDIO_API_BASE	http://localhost:1234/v1	LM Studio local LLM (preferred)
LLM_MODEL	lm_studio/producer/model-name	Model name
GEMINI_API_KEY	your-gemini-key	Auto-selects gemini/gemini-2.0-flash
OPENAI_API_KEY	sk-...	Auto-selects gpt-4o
ANTHROPIC_API_KEY	sk-ant-...	Auto-selects anthropic/claude-sonnet-4-20250514

Using the Web App

1. Paste a video URL — YouTube, TikTok, Instagram, or any supported site.
2. Add instructions (optional) — e.g., "Focus on the funniest moments, make 3 shorts".
3. Click "Create Shorts" — The agent downloads the video, detects highlights via optical flow, picks the best clips, reframes for vertical, applies effects, and exports.
4. Download your shorts — Exported files appear in the "Exported Shorts" panel with one-click download buttons.

Note: LLM settings can be configured via environment variables (recommended) or overridden in the UI's "LLM Settings" panel. Environment auto-detection means zero config for most setups.

---

⚙️ How It Works

sequenceDiagram
    participant User
    participant WebUI as AI Shorts Creator<br/>(NiceGUI)
    participant YTD as yt-dlp
    participant LLM as LLM Provider<br/>(LM Studio / Cloud)
    participant MCP as vidmagik-mcp<br/>Backend

    User->>WebUI: Paste URL + optional instructions
    WebUI->>YTD: Download source video
    YTD-->>WebUI: Local file path

    WebUI->>LLM: System prompt + video path
    loop Agentic Loop (up to 50 iterations)
        LLM-->>WebUI: Tool call request
        WebUI->>MCP: Execute tool via MCP
        MCP-->>WebUI: Tool result
        WebUI->>LLM: Feed result back
    end
    LLM-->>WebUI: Final summary of creative decisions
    WebUI-->>User: Download links for finished shorts

The Agentic Loop

The heart of the project is the agentic loop in src/app/mcp_client.py. Here's what happens when you click "Create Shorts":

1. Download — yt-dlp downloads the video to the media/ directory.
2. System Prompt — The app sends a system prompt to the configured LLM, instructing it to act as an expert video editor.
3. Tool Calling Loop — The LLM decides what to do and calls MCP tools:
   • video_file_clip → Load the downloaded video
   • tools_detect_highlights → Optical-flow analysis to find high-motion moments
   • subclip → Extract the best 5–15 second segments around each highlight
   • vfx_auto_framing → Crop to 9:16 vertical with face tracking
   • concatenate_video_clips → Combine all clips into one highlight reel
   • vfx_fade_in / vfx_fade_out → Smooth transitions
   • write_videofile → Export to media/short.mp4
4. Summary — The LLM explains its creative choices (why it picked those moments, what effects it applied).
5. UI Updates — Every tool call and result streams to the Agent Activity log in real time.

Architecture Overview

vidmagikAgent/
├── src/                             # 📦  SOURCE PACKAGE
│   ├── __init__.py                  #    Package marker
│   ├── main.py                      #    CLI entry point (launches web app)
│   ├── inspect_moviepy.py           #    MoviePy installation checker
│   │
│   ├── api/                         # ⚙️  BACKEND
│   │   ├── __init__.py              #    Package init
│   │   ├── main.py                  #    vidmagik-mcp server (60+ MCP tools, prompts, upload route)
│   │   └── custom_fx/               #  🎨  EFFECTS LIBRARY
│   │       ├── __init__.py           #    Re-exports all effects + highlight detection
│   │       ├── auto_framing.py       #    Face-tracking vertical crop
│   │       ├── chroma_key.py         #    Green screen removal
│   │       ├── clone_grid.py         #    Grid of video clones
│   │       ├── highlight_detect.py   #    Optical-flow highlight detection
│   │       ├── kaleidoscope.py       #    Radial symmetry
│   │       ├── kaleidoscope_cube.py  #    Kaleidoscope + rotating cube combo
│   │       ├── matrix.py             #    "Matrix" digital rain overlay
│   │       ├── quad_mirror.py        #    Four-quadrant mirror
│   │       ├── rgb_sync.py           #    RGB channel split / glitch
│   │       └── rotating_cube.py      #    3D rotating cube with video mapping
│   │
│   └── app/                         # 🖥  FRONTEND — AI Shorts Creator
│       ├── __init__.py              #    Package init
│       ├── main.py                  #    NiceGUI web UI (dark mode, real-time log)
│       ├── mcp_client.py            #    MCP client + LLM agentic loop (LiteLLM)
│       └── Dockerfile               #    Frontend Docker image
│
├── tests/                           # 🧪  TEST SUITE
│   ├── test_e2e.py                  #    Backend end-to-end tests
│   ├── test_nicegui_integration.py  #    NiceGUI integration tests
│   └── frontend_e2e_test.py         #    Frontend end-to-end tests
│
├── media/                           # 📁  Working directory (gitignored)
├── Dockerfile                       # 🐳  Backend Docker image
├── docker-compose.yml               # 🐳  Multi-service orchestration
├── pyproject.toml                   # 📦  Dependencies, packaging & CLI config
├── uv.lock                          # 🔒  Locked dependency versions
├── .env.example                     # 🔑  Environment variable template
├── CUSTOM_FX.md                     # 📖  Custom effects documentation
└── LICENSE                          # ⚖️  MIT License

---

🖥 Web App

UI Sections

The AI Shorts Creator (src/app/main.py) is a dark-themed NiceGUI single-page application with four main sections:

1. LLM Settings (Collapsible)

Configure the LLM backend. Can be preset via environment variables or adjusted in the UI:

Field	Env Var	Notes
API Base URL	LM_STUDIO_API_BASE	Any OpenAI-compatible endpoint
API Key	LM_STUDIO_API_KEY	Auto-detected from provider env vars
Model	LLM_MODEL	ibm/granite-4-h-tiny, gpt-4o, gemini-2.0-flash, etc.

2. Video Source

• Video URL — Paste any URL supported by yt-dlp.
• Instructions (optional) — Natural language creative direction for the agent.
• "Create Shorts" button — Kicks off the entire pipeline.

3. Agent Activity Log

A scrolling, chat-style log that streams the agent's work in real time:

• 🧠 Thinking — The LLM's reasoning (gray)
• 🔧 Tool Calls — Which MCP tool is being called and with what arguments (violet)
• ✅ Tool Results — The result of each tool call (cyan)
• 💬 Messages — The LLM's final summary (green)
• ❌ Errors — Any failures (red)

4. Exported Shorts

Download cards for each exported video file, with movie icon and one-click download buttons.

MCP Client (src/app/mcp_client.py)

The MCPVideoClient class handles the full lifecycle:

• Connection — Connects to the vidmagik-mcp backend via StreamableHTTP (when MCP_SERVER_URL is set, e.g. in Docker) or spawns it as a local subprocess over stdio transport.
• Schema Discovery — Fetches all MCP tool schemas and converts them to OpenAI function-calling format.
• Video Download — Uses yt-dlp to download videos with best quality MP4 format.
• Agentic Loop — Iterates up to 50 rounds of LLM → tool call → result → LLM, yielding typed events for the UI to render.
• LiteLLM Integration — Translates MCP tool schemas to OpenAI format so any OpenAI-compatible model can drive the agent.
• Auto-Config — Resolves LLM provider automatically from environment variables (LM Studio → Gemini → OpenAI → Anthropic).

---

⚙️ MCP Backend

The backend (src/api/main.py) is a FastMCP server that wraps the entire MoviePy video editing library as MCP tools. It can run in three transport modes:

# HTTP  — for standalone use or StreamableHTTP MCP clients
uv run src/api/main.py --transport http --host 0.0.0.0 --port 8080

# SSE — for Server-Sent Events MCP clients
uv run src/api/main.py --transport sse

# stdio (default)— for subprocess-based MCP clients (used by the AI Shorts Creator locally)
uv run src/api/main.py --transport stdio

Clip Management System

All clips live in an in-memory store (CLIPS dict) with a max of 100 concurrent clips. Every tool that creates or transforms a clip returns a UUID handle:

video_file_clip("video.mp4") → "a1b2-..."
subclip("a1b2-...", 10.0, 25.0) → "c3d4-..."
vfx_fade_in("c3d4-...", 1.0) → "e5f6-..."
write_videofile("e5f6-...", "output.mp4") → "Successfully wrote video to output.mp4"

Full Tool Inventory (60+ tools)

Clip Management (5 tools)

Tool	Description
validate_path(filename)	Path validation to prevent directory traversal
register_clip(clip)	Register a clip and return its UUID
get_clip(clip_id)	Retrieve a clip by UUID
list_clips()	List all loaded clips and their types
delete_clip(clip_id)	Remove a clip from memory

Video I/O (10 tools)

Tool	Description
video_file_clip(filename, audio, fps_source, target_resolution)	Load a video file
image_clip(filename, duration, transparent)	Load an image as a clip
image_sequence_clip(sequence, fps, durations, with_mask)	Create clip from image sequence or folder
text_clip(text, font, font_size, color, bg_color, size, method, duration)	Create a text clip (needs ImageMagick)
color_clip(size, color, duration)	Create a solid color clip
credits_clip(creditfile, width, color, stroke_color, ...)	Scrolling credits from a text file
subtitles_clip(filename, encoding, font, font_size, color)	Subtitles from .srt file
write_videofile(clip_id, filename, fps, codec, audio_codec, bitrate, preset, ...)	Export video to file
write_gif(clip_id, filename, fps, loop)	Export clip as GIF
tools_ffmpeg_extract_subclip(filename, start_time, end_time, targetname)	Fast FFmpeg extraction (no decoding)

Audio I/O (2 tools)

Tool	Description
audio_file_clip(filename, buffersize)	Load an audio file
write_audiofile(clip_id, filename, fps, nbytes, codec, bitrate)	Export audio to file

Clip Configuration (6 tools)

Tool	Description
set_position(clip_id, x, y, pos_str, relative)	Set clip position
set_audio(clip_id, audio_clip_id)	Set video's audio track
set_mask(clip_id, mask_clip_id)	Set transparency mask
set_start(clip_id, t)	Set start time
set_end(clip_id, t)	Set end time
set_duration(clip_id, t)	Set duration

Compositing & Arrangement (6 tools)

Tool	Description
subclip(clip_id, start_time, end_time)	Cut a portion of a clip
composite_video_clips(clip_ids, size, bg_color, use_bgclip)	Overlay/compose clips
tools_clips_array(clip_ids_rows, bg_color)	Arrange clips in a grid
concatenate_video_clips(clip_ids, method, transition)	Concatenate clips
composite_audio_clips(clip_ids)	Mix audio clips
concatenate_audio_clips(clip_ids)	Concatenate audio clips

Built-in Video Effects (32 tools)

Tool	Description
vfx_accel_decel	Accelerate / decelerate playback
vfx_black_white	Convert to black and white
vfx_blink	Blink on/off
vfx_crop	Crop region
vfx_cross_fade_in / vfx_cross_fade_out	Cross-fade transitions
vfx_even_size	Ensure even pixel dimensions
vfx_fade_in / vfx_fade_out	Fade from/to black
vfx_freeze / vfx_freeze_region	Freeze frame or region
vfx_gamma_correction	Adjust gamma
vfx_head_blur	Blur a moving point (math expressions)
vfx_invert_colors	Invert colors
vfx_loop	Loop a clip
vfx_lum_contrast	Luminosity & contrast
vfx_make_loopable	Seamless loop with fade
vfx_margin	Add border/margin
vfx_mask_color	Create mask from color
vfx_masks_and / vfx_masks_or	Logical mask operations
vfx_mirror_x / vfx_mirror_y	Mirror horizontally/vertically
vfx_multiply_color	Color intensity
vfx_multiply_speed	Playback speed
vfx_painting	Oil painting effect
vfx_resize	Resize clip
vfx_rotate	Rotate clip
vfx_scroll	Scrolling viewport
vfx_slide_in / vfx_slide_out	Slide transitions
vfx_supersample	Anti-aliasing
vfx_time_mirror	Reverse playback
vfx_time_symmetrize	Play forward then reverse

Custom Video Effects (9 tools)

Tool	Description
vfx_auto_framing(clip_id, target_aspect_ratio, smoothing)	Face-tracking vertical crop
vfx_chroma_key(clip_id, color, threshold, softness)	Green screen removal
vfx_clone_grid(clip_id, n_clones)	Grid of video clones
vfx_kaleidoscope(clip_id, n_slices, x, y)	Radial symmetry
vfx_kaleidoscope_cube(clip_id, kaleidoscope_params, cube_params)	Combined kaleidoscope + cube
vfx_matrix(clip_id, speed, density, chars, color, font_size)	Matrix digital rain
vfx_quad_mirror(clip_id, x, y)	Four-quadrant mirror
vfx_rgb_sync(clip_id, r/g/b_offset, r/g/b_time_offset)	RGB channel split glitch
vfx_rotating_cube(clip_id, speed, direction, zoom)	3D rotating cube

Audio Effects (7 tools)

Tool	Description
afx_audio_delay(clip_id, offset, n_repeats, decay)	Echo / delay
afx_audio_fade_in / afx_audio_fade_out	Fade in/out
afx_audio_loop(clip_id, n_loops, duration)	Loop audio
afx_audio_normalize(clip_id)	Normalize levels
afx_multiply_stereo_volume(clip_id, left, right)	Stereo balance
afx_multiply_volume(clip_id, factor)	Volume control

Analysis & Utility (8 tools)

Tool	Description
tools_detect_scenes(clip_id, luminosity_threshold)	Detect scene boundaries
tools_detect_highlights(clip_id, threshold)	Optical-flow highlight detection — finds high-motion moments
tools_find_video_period(clip_id, start_time)	Find video repeating period
tools_find_audio_period(clip_id)	Find audio repeating period
tools_drawing_color_gradient(size, p1, p2, col1, col2, shape, offset)	Generate gradient image
tools_drawing_color_split(size, x, y, p1, p2, col1, col2, grad_width)	Generate color split image
tools_file_to_subtitles(filename, encoding)	Parse .srt subtitle file
tools_check_installation()	Verify MoviePy + FFmpeg install

Prompt Templates

Pre-built prompt presets for common creative workflows:

Prompt	Use Case
demonstrate_kaleidoscope(clip_id)	8-slice kaleidoscope for psychedelic visuals
glitch_effect_preset(clip_id)	High-energy RGB split glitch for music videos
matrix_intro_preset(clip_id)	Matrix digital rain for tech/hacker intros
auto_framing_for_tiktok(clip_id)	Vertical 9:16 with face tracking
rotating_cube_transition(clip_id)	3D cube scene transition
slideshow_wizard(images, ...)	Slideshow with transitions, text overlays, configurable resolution
title_card_generator(text, ...)	Title card with solid background and typography
demonstrate_kaleidoscope_cube(clip_id, ...)	Combined kaleidoscope + cube demo

File Upload API

The backend exposes a custom HTTP endpoint for direct file uploads:

curl -X POST http://localhost:8080/upload -F "file=@/path/to/video.mp4"
# → {"filename": "/app/video.mp4", "size": 12345678}

---

🎨 Custom Effects

All 9 custom effects are in src/api/custom_fx/ and follow the MoviePy Effect protocol. The module also includes a standalone highlight detection function. Full parameter documentation is in CUSTOM_FX.md.

Effect	File	What It Does
Auto Framing	auto_framing.py	Face-tracking crop for vertical video. Uses Haar Cascades + exponential smoothing for cinematic tracking. Ideal for landscape → portrait conversion.
Matrix Rain	matrix.py	Falling character overlay with bright leading edge and fading trails. Configurable speed, density, charset, color.
Kaleidoscope	kaleidoscope.py	Radial symmetry — mirrors a wedge around a center point. Configurable slice count and center.
RGB Sync	rgb_sync.py	Splits RGB channels with independent spatial (px) and temporal (seconds) offsets. Chromatic aberration / glitch aesthetic.
Chroma Key	chroma_key.py	Green screen removal via Euclidean distance masking. Configurable threshold and softness.
Clone Grid	clone_grid.py	Tiles video in auto-calculated grid layout. Supports 2–64+ clones.
Quad Mirror	quad_mirror.py	Four-quadrant symmetry around a configurable center point.
Rotating Cube	rotating_cube.py	3D cube with video on all faces. Multi-axis rotation, optional quad-mirroring, circular motion paths.
KaleidoscopeCube	kaleidoscope_cube.py	Compound effect: Kaleidoscope → Rotating Cube with independent config for each stage.
Highlight Detection	highlight_detect.py	Optical-flow (Farneback) analysis on keyframes to detect high-motion moments. Returns timestamped highlights with intensity scores.

---

🐳 Docker

One-Command Deployment

# Build and start everything
docker compose up --build

# Just the web app (connects to MCP backend via StreamableHTTP)
docker compose up shorts-creator

# Just the MCP backend standalone
docker compose up vidmagik-mcp

Services

Service	Description	Port	Dockerfile
shorts-creator	AI Shorts Creator web app	3000	src/app/Dockerfile
vidmagik-mcp	MCP server (HTTP mode)	8080	Dockerfile

In Docker Compose, the frontend connects to the backend over StreamableHTTP via the MCP_SERVER_URL environment variable (set to http://vidmagik-mcp:8080/mcp). When running locally without Docker, the frontend spawns the backend as a local subprocess over stdio.

Docker Details

Backend image (Dockerfile):

• Base: python:3.12-slim
• System deps: ffmpeg, imagemagick, libsm6, libxext6, libgl1
• Auto-patches ImageMagick policy for TextClip support
• Package manager: uv (install + sync from lockfile, pinned to Python 3.13)

Frontend image (src/app/Dockerfile):

• Base: python:3.12-slim
• System deps: ffmpeg
• Pinned to Python 3.13
• Exposes port 3000

Shared volume: ./media:/app/media — both services share video files.

Environment Variables

Variable	Default	Description
PYTHONUNBUFFERED	1	Disable output buffering
NICEGUI_HOST	127.0.0.1	NiceGUI bind address (0.0.0.0 in Docker)
MCP_SERVER_URL	—	StreamableHTTP URL for remote MCP server (Docker only)
LM_STUDIO_API_BASE	—	LM Studio API endpoint
LLM_MODEL	—	Model name (auto-prefixed for LM Studio)
GEMINI_API_KEY	—	Gemini API key (auto-selects model)
OPENAI_API_KEY	—	OpenAI API key (auto-selects model)
ANTHROPIC_API_KEY	—	Anthropic API key (auto-selects model)

---

🧪 Testing

Test Suite

File	Scope
tests/test_e2e.py	Backend MCP tools — end-to-end tests
tests/test_nicegui_integration.py	NiceGUI UI component integration tests
tests/frontend_e2e_test.py	Full frontend end-to-end tests

Running Tests

# All tests
uv run pytest

# With coverage
uv run pytest --cov=src --cov-report=term-missing

# Specific module
uv run pytest tests/test_e2e.py -v

Pytest config (pyproject.toml):

[tool.pytest.ini_options]
asyncio_mode = "auto"

---

📦 Dependencies

Runtime

Package	Purpose
FastMCP ≥ 3.0.0	MCP server framework
MoviePy ≥ 2.2.1	Video editing engine
LiteLLM ≥ 1.40.0	Universal LLM API client (LM Studio, Gemini, OpenAI, Anthropic, local, etc.)
NiceGUI ≥ 2.0.0	Web UI framework for the Shorts Creator
OpenCV (headless) ≥ 4.13.0	Computer vision — face detection for auto-framing, optical flow for highlight detection
NumExpr ≥ 2.14.1	Safe math expression evaluation (HeadBlur effect)
yt-dlp ≥ 2024.0.0	Video downloading from 1000+ sites

Dev

Package	Purpose
pytest ≥ 9.0.2	Testing framework
pytest-asyncio ≥ 1.3.0	Async test support
pytest-cov ≥ 7.0.0	Coverage reporting
httpx ≥ 0.28.1	HTTP client for testing

System

• FFmpeg — Required by MoviePy for all encoding/decoding
• ImageMagick — Required for text rendering (TextClip)

---

🛠 Development

Setting Up for Development

# Clone the repo
git clone https://github.com/vizionik25/vidmagikAgent.git
cd vidmagikAgent

# Install all dependencies (including dev tools)
uv sync

Running from Source

# Launch the web app
uv run src/app/main.py
# → Opens at http://127.0.0.1:3000

Running the MCP Server Standalone

# StreamableHTTP mode (for remote clients, Docker, or browser-based tools)
uv run src/api/main.py --transport http --host 0.0.0.0 --port 8080

# SSE mode (Server-Sent Events)
uv run src/api/main.py --transport sse --host 0.0.0.0 --port 8080

# stdio mode (default — for subprocess-based MCP clients)
uv run src/api/main.py --transport stdio

Server flags:

Flag	Default	Description
--transport	http	Transport protocol: stdio, sse, or http
--host	0.0.0.0	Bind address for HTTP/SSE modes
--port	8080	Port for HTTP/SSE modes

Running Tests

# All tests
uv run pytest

# With coverage report
uv run pytest --cov=src --cov-report=term-missing

# Specific test file
uv run pytest tests/test_e2e.py -v

Uploading Files to the MCP Server

When the server is running in HTTP mode, you can upload files directly:

curl -X POST http://localhost:8080/upload -F "file=@/path/to/video.mp4"
# → {"filename": "/app/video.mp4", "size": 12345678}

Adding a New Custom Effect

1. Create src/api/custom_fx/my_effect.py:

from moviepy import Effect
import numpy as np

class MyEffect(Effect):
    """Description of your effect."""

    def __init__(self, intensity: float = 1.0):
        self.intensity = intensity

    def apply(self, clip):
        def filter(get_frame, t):
            frame = get_frame(t)
            # Transform the frame here
            return frame
        return clip.transform(filter)

2. Export from src/api/custom_fx/__init__.py:

from .my_effect import MyEffect # add your effect here 

3. Add MCP tool in src/api/main.py:

from custom_fx import MyEffect # be sure to add your effect to the custom_fx import

# The tool name must start with vfx_ and end with _effect (this is how the frontend knows it's a video effect)
# The tool must return the clip_id of the modified clip
# The MCP Server File is quite large so please keep your orginization clean it makes it easier to find your code

@mcp.tool
def vfx_my_effect(clip_id: str, intensity: float = 1.0) -> str:
    """Description of your custom effect."""
    clip = get_clip(clip_id)
    return register_clip(clip.with_effects([MyEffect(intensity)]))

---

📄 License

MIT License — Copyright © 2026 vizionik25