manim-engine

A minimal, aesthetic CLI engine that turns a JSON content plan into a verified physics education video rendered with Manim. It is built for agents — an AI agent authors the plan (content, math, layout, expected visuals), the engine does the mechanical work (render, assemble, verify), and returns a machine-readable report with an exit code so the agent never has to eyeball frames.

$ manim-engine doctor
$ manim-engine plan examples/ring-disc/plan.json
$ manim-engine build examples/ring-disc/plan.json --draft     # fast iteration
$ manim-engine build examples/ring-disc/plan.json             # final 1080p60
$ manim-engine verify .engine/ring-disc.mp4 examples/ring-disc/plan.json

manim-engine is a single command installed on PATH — no absolute paths, no working-directory requirements; run it from anywhere. Exit codes: 0 = everything ok (including all frame checks), 1 = any error or failed check. Append --json for a single JSON object on stdout.

---

1. What this engine is (and is not)

Is: a deterministic pipeline plan → validate → per-segment manim render → concat → frame verification → report. Content correctness lives in the plan, which the agent verifies before rendering (the ledger field is the place to record every symbol, value and formula used). Rendering and checking are mechanical and repeatable.

Is not: a Manim IDE, a physics solver, or a generic video editor. It produces educational/physics-style videos: title cards, question statements, equations, animated diagrams, boxed answers. If a segment needs bespoke animation (a race, a force diagram), it is supplied as a plain Manim Scene class — see Custom scenes.

Why verification matters: agents cannot reliably "watch" a 74-second video, but they can compare numeric pixel facts against expectations they calibrated themselves: "at the end of the race scene the blue disc centroid must be within 12 px of (919, 857)", "the yellow ring must occupy ≥ 400 pixels there". build fails (exit 1) if any check fails, so an agent gets feedback it can actually act on.

Reference example: examples/ring-disc/ — JEE Advanced 2018, ring vs disc rolling down a 60° incline. The physics was SymPy-verified first, then rendered as a 74.4 s, 1920×1080@60 video with 4 frame checks, all passing.

---

2. Quick start

Install (one command)

Windows:

irm https://<host>/install.ps1 | iex

macOS / Linux:

curl -fsSL https://<host>/install.sh | sh

That one command downloads the engine, creates an isolated .venv, installs every Python dependency (manim, numpy, a bundled static ffmpeg via imageio-ffmpeg), and auto-installs missing system tools (ffmpeg / LaTeX) via winget / brew / apt — then runs doctor to prove it works. Prefer uv for speed if present.

Alternatively, without hosting, anyone with the repo can run the same script locally:

# from the repo root
sh install.sh            # macOS / Linux
powershell -ExecutionPolicy Bypass -File .\install.ps1   # Windows

Or do it manually — all Python packages come from one command:

pip install .            # or: uv pip install . | pipx install . | uv tool install .
manim-engine doctor --fix   # installs ffmpeg + LaTeX via winget/brew/apt if missing

Only LaTeX (for math rendering) is a system binary that pip/uv cannot ship — doctor --fix handles it. After any system-tool install, open a new terminal so PATH updates take effect.

Use

# 1. environment check (add --fix to auto-install missing tools)
manim-engine doctor --fix

# 2. write a plan skeleton and fill it in
manim-engine new my-plan.json

# 3. validate + summarize
manim-engine plan my-plan.json

# 4. iterate fast: half resolution, 15 fps (~5x faster)
manim-engine build my-plan.json --draft

# 5. final quality
manim-engine build my-plan.json -o out/my-video.mp4

# 6. re-verify any built video against the plan
manim-engine --json verify out/my-video.mp4 my-plan.json

--json is a global flag — it goes before the subcommand (manim-engine --json build ...). Without installation, the same commands work from the repo root as python engine.py ... or python -m engine ....

---

3. The agent workflow (the intended loop)

1. Plan the content (agent-side, off-engine): derive every formula and value, and record them in the plan's ledger. For the ring-disc video this was done with SymPy before any rendering.
2. Author plan.json: segments in order; per-segment JSON fields control timing, colors, and layout. Use expected_duration as your pacing budget; treat it as a plan — the build reports actuals.
3. Calibrate checks (agent-side, once): build with --draft, extract frames at interesting moments (engine exports a frame-extraction helper, engine.verify._extract_rgb_frame), measure the blue-centroid / yellow-count / density you expect, and encode those numbers as checks with segment-relative times.
4. Run the loop: build --draft → fix plan/custom scenes until exit 0 → build at full quality → verify to confirm.
5. Consume the report: --json gives {ok, video, duration, segments[{id,duration}], checks[{at,type,ok,measured,expected}]}.

How an agent invokes it

manim-engine is a plain console command on PATH, so a calling agent just shells out — no absolute paths, no working directory requirements:

manim-engine build path\to\plan.json --json      # exit 0/1 + JSON report on stdout
manim-engine verify video.mp4 plan.json --json   # re-verify after the fact

A plan directory is self-contained and portable: segment content, custom scene modules, and checks all resolve relative to the plan file (absolute paths also accepted). Share plan.json + its custom scenes and anyone can rebuild the identical video.

---

4. Plan schema (complete reference)

{
  "id": "ring-disc",                 // required, non-empty; used for file naming
  "title": "JEE Advanced 2018 — Ring vs Disc on an Incline",
  "resolution": [1920, 1080],        // optional, default [1920, 1080]
  "fps": 60,                         // optional, default 60
  "style": {                         // optional; named/hex colors, see §4.1
    "heading": "yellow", "accent": "blue", "ink": "white",
    "muted": "gray", "green": "green", "orange": "orange", "red": "red"
  },
  "ledger": [                        // optional; documented facts (no render effect)
    { "symbol": "a_ring", "meaning": "ring acceleration",
      "rendered": "(g sin theta)/2 = 5√3/2 ≈ 4.33" }
  ],
  "checks": [                        // optional; frame checks, see §6
    { "seg": "race", "at": 9.1, "type": "centroid",
      "color": "blue", "expected": [919, 857], "tol": 12 },
    { "seg": "race", "at": 9.1, "type": "count",
      "color": "yellow", "min_count": 400 }
  ],
  "segments": [ /* see §5 */ ]
}

4.1 Style and colors

style keys: heading, accent, ink, muted, green, orange, red (defaults: yellow, blue, white, gray, green, orange, red). Values are named colors (yellow, blue, white, gray/grey, green, orange, red), #RRGGBB hex strings, or raw manim color objects in custom scenes. Individual segments/lines may override with their own color fields.

---

5. Segment kinds

5.1 title — heading + subtitle lines

{
  "id": "title", "kind": "title",
  "lines": [
    { "role": "heading", "text": "JEE ADVANCED 2018", "size": 64 },
    { "role": "sub", "text": "Paper 1 · Physics · Q9 (Numerical)", "size": 34 }
  ],
  "heading_color": "yellow",      // optional
  "sub_color": "gray",            // optional
  "final_wait": 2.0               // optional, default 2.0
}

Heading is Writen over 3 s; each sub-line FadeIns with an upward shift.

5.2 text — sequential blocks (question statements, narration lines)

{
  "id": "question", "kind": "text",
  "blocks": [
    { "text": "A ring and a disc are initially at rest, side by side,\nat the top of an inclined plane that makes an angle 60°\nwith the horizontal.",
      "run_time": 1.6, "pause": 0.8 },
    { "text": "If the time difference between their reaching the ground is",
      "run_time": 1.6, "pause": 1.2,
      "with": [ { "kind": "math", "latex": "\\frac{2-\\sqrt{3}}{\\sqrt{10}}", "size": 38 } ] }
  ],
  "top_buff": 0.7,                // optional
  "block_buff": 0.9,              // optional, spacing between blocks
  "final_wait": 1.5               // optional, default 1.5
}

\n inside text makes newlines. with attaches extra mobjects (inline math, e.g. the value of a fraction) to a block; inline items may set kind, latex/text, color, size.

5.3 math — stacked, left-aligned equations

{
  "id": "kinematics", "kind": "math",
  "note": { "text": "Kinematics", "size": 28 },   // optional top caption
  "lines": [
    { "latex": "s = \\frac{h}{\\sin\\theta}, \\qquad t = \\sqrt{\\frac{2s}{a}}",
      "run_time": 1.0, "pause_after": 1.0 },
    { "latex": "t_{\\text{ring}} = \\sqrt{\\frac{16h}{3g}}",
      "color": "yellow", "run_time": 1.0 },
    { "latex": "t_{\\text{disc}} = \\sqrt{\\frac{4h}{g}}", "color": "blue",
      "run_time": 1.0, "pause_after": 1.5,
      "with": [ { "kind": "math", "latex": "\\left(\\sin^2 60^\\circ = \\frac{3}{4}\\right)" } ] }
  ],
  "top_buff": 1.0,                // optional, default 1.0
  "line_buff": 0.6,               // optional, default 0.6
  "final_wait": 0.0               // optional, default 2.0
}

Each line is Writen at run_time (default 1.0), then the video pauses pause_after (default 0). Use with for side annotations rendered next to a line.

5.4 answer — equation stack + boxed answer + takeaway

{
  "id": "answer", "kind": "answer",
  "lines": [
    { "latex": "(2-\\sqrt{3})\\sqrt{\\frac{2h}{15}} = \\frac{2-\\sqrt{3}}{\\sqrt{10}}" },
    { "latex": "\\sqrt{\\frac{2h}{15}} = \\frac{1}{\\sqrt{10}} \\;\\Rightarrow\\; h = \\frac{3}{4}" }
  ],
  "answer": { "latex": "h = 0.75\\ \\text{m}", "color": "yellow" },
  "takeaway": "The disc wins — less rotational inertia means faster translation.",
  "top_buff": 1.4,                // optional, default 1.4
  "final_wait": 2.5               // optional, default 2.5
}

Lines are Writen (1.0 s each + 1.2 s pause); the answer is Writen and enclosed in a SurroundingRectangle; the takeaway fades in at the bottom.

5.5 custom — your own Manim scene

{
  "id": "race", "kind": "custom",
  "module": "race_scene.py",   // path relative to the plan dir (or absolute)
  "class": "RaceScene",
  "expected_duration": 18.6
}

The engine loads the file, instantiates the class, and renders it like any other segment (it participates in concat and checks). See §7 for the scene contract and known Manim-0.20.1 gotchas.

---

6. Checks (frame verification)

Checks run against the final concatenated video at times that are seg-relative and resolved to absolute times using the actual rendered segment durations (from the build manifest; standalone verify falls back to expected_duration if no manifest exists next to the video). All checks are resolution-scaled, so the same plan works at draft (960×540) and full (1920×1080) resolution.

type	fields	semantics
density	min_fraction (default 0.001)	fraction of frame pixels darker than luminance 140 must be ≥ min_fraction. Catches "empty/black scene" regressions.
centroid	color (default blue), expected [x, y] at plan resolution, tol (default 10)	centroid of all pixels of that color within tol px (both scaled to actual resolution). Verifies where something is.
count	color (default yellow), min_count	number of pixels of that color (reported as full-resolution-equivalent) must be ≥ min_count. Verifies that something is present.

Color classifiers (validated on the reference video; the same thresholds power _RULES in engine/verify.py):

color	predicate (r, g, b)
yellow	r>190, g>190, b<120
blue	b>140, r<110
red	r>180, g<110, b<110
orange	r>200, 90<g<180, b<80
green	g>170, g>r+40, g>b+40
white	r>235, g>235, b>235

Density sampling: every 2nd pixel on both axes (1/4 of the frame); count/centroid use the same sample grid but counts are scaled back to full-resolution equivalents.

Calibration workflow (how an agent sets expected/tol/min_count)

1. Render with --draft.
2. Extract the frame at the interesting moment (e.g. engine.verify._extract_rgb_frame(video, t) returns (w, h, rgb_bytes)), or via ffmpeg -ss T -i video.mp4 -frames:v 1 out.png.
3. Measure the centroid/count with _frame_stats(w, h, data)["px"]["blue"] etc.
4. Encode the measured value (at plan resolution) plus a sensible tolerance, and the count threshold (e.g. 400 for a fully-visible ring), into the plan.

Reference values from examples/ring-disc/plan.json (all currently passing): disc centroid at race end measured (918.5, 857.2) vs expected (919, 857), tol 12; ring yellow count 908 vs min 400.

---

7. Custom scenes contract

• The file is a plain Manim module. class RaceScene(Scene) with construct(self).
• The engine imports the class by name and renders it with manim --disable_caching --media_dir <work>/media/<seg> --resolution W,H --fps N <generated> <ClassName>Scene.
• Use the plan style colors if you want consistency: from engine.style import resolve_style / resolve_color, or hardcode manim colors (the examples do).
• The full frame is 1920×1080 px; Manim's default frame height is 8 scene units → 135 px per unit, width 14.22 units, origin at screen center. Screen pixel: px_x = (scene_x + 7.11) * 135, px_y = (3.98 - scene_y) * 135.

Manim 0.20.1 gotchas (all already fixed inside the examples)

• ACCENT is not exported — use BLUE.
• Mobjects have no .create — use Create(mobj).
• wait(0) raises — guard every wait with if duration > 0.
• Remember Write, LEFT, etc. imports; the engine's generated files only import what the builders need.

---

8. Pipeline internals

build:
  load plan (validated)                        engine/plan.py
  └ per segment:
      generate_segment → <work>/generated/<id>.py   engine/render.py
        - standard kinds: subclass of builder_for(seg), SEGMENT = raw JSON
        - custom: subclass of load_custom(module, class)
      render_segment → manim subprocess → media dir → latest mtime mp4
  concat → ffmpeg -f concat -safe 0 -c copy    (stream copy, same codec)
  write <work>/<id>.manifest.json               (actual per-segment durations)
  checks → resolved to absolute times → per-check results
  report → human lines or --json doc            engine.py

Layout under the plan directory (defaults): <plan_dir>/.engine/ for generated code and media, .engine/<id>.mp4 for the video, .engine/<id>.manifest.json for durations.

Performance (measured, ring-disc example)

mode	resolution	fps	render time (6 segments, 74.4 s video)
--draft	960×540	15	~23 s
full	1920×1080	60	~77 s

---

9. Known limitations / roadmap

• MathTex requires a working LaTeX (pdflatex) installation.
• Frame checks sample 1/4 of pixels; fine for large marks, weak for 1–2 px details (they are still detected — the ring's 8 px stroke counts 908 px full-res).
• No audio track yet — narration/TTS, subtitles, and a segment-kind catalog are natural next steps.
• expected_duration is a planning aid, not enforced; if you need hard pacing, tighten the final report comparison (engine reports both planned and actual).
• LaTeX (pdflatex) is required for MathTex and cannot be installed by pip/uv — use manim-engine doctor --fix (or the bootstrap scripts) which install it via winget/brew/apt.

---

10. Repository map

manim-engine/
├── engine.py              thin shim: `python engine.py ...` works without install
├── install.sh/.ps1        one-command installers (curl | sh  /  irm | iex)
├── bootstrap.ps1/.sh      scripted venv + deps + ffmpeg + LaTeX setup
├── pyproject.toml         package metadata + `manim-engine` console command
├── engine/
│   ├── cli.py             CLI entry: doctor(--fix) | plan | new | build | verify, --json, exit codes
│   ├── plan.py            plan loading + validation
│   ├── style.py           named/hex color resolution
│   ├── tools.py           ffmpeg/ffprobe resolution (PATH → imageio-ffmpeg fallback), duration/dims probing
│   ├── scenes.py          TitleBuilder, TextBuilder, MathBuilder, AnswerBuilder, load_custom
│   ├── render.py          generate_segment, render_segment (manim), concat (ffmpeg)
│   └── verify.py          frame extraction, _frame_stats, run_check
└── examples/ring-disc/
    ├── plan.json          the reference plan (6 segments, ledger, 4 checks)
    ├── race_scene.py      custom scene: ring vs disc race + geometry annotations
    └── accel_scene.py     custom scene: accelerations + force diagram

The reference physics facts (SymPy-verified, recorded in the ledger): a_ring = (g sinθ)/2 = 5√3/2 ≈ 4.330, a_disc = (2/3) g sinθ = 10√3/3 ≈ 5.774, t_ring = √(16h/3g), t_disc = √(4h/g), time difference (2−√3)√(2h/15), answer h = 3/4 = 0.75 m.