Beatoven

Compose hierarchical animation & simulation plans, generate full library-backed code, play it back, and inline-edit with .edit.

Beatoven turns a prompt into a playable AnimationObject:

• Imports full capability surfaces across JS/Python/Rust/C#/Java — GSAP, Three.js, Babylon, path tracers, Manim, Bevy, Unity, Unreal, Godot, O3DE, Flax, and more
• Max engine suites — import entire Unity / Unreal / OSS surfaces for real-time asset creation beyond Babylon (max_feature=True)
• Image guide — local Hugging Face (or heuristic) concept plates → physical grounding → hierarchical anim/sim scaffold
• SLM / cloud guided compose binds real library APIs via heuristic, local Qwen, or OpenAI/Claude/Grok
• Builds a hierarchy plan: scene → acts → shots → layers → tracks → keyframes
• anim.play(playback=…) previews whatever was generated (web / python / IR / unity / unreal / …)
• anim.edit / .deep / .cascade / .image_guide / .deepen_guide for iterative enhancement
• Optional Hugging Face local SLM, diffusers image gen, and on-device 3D ladders (TripoSR → SF3D → Hunyuan)

Install

From PyPI (recommended)

pip install beatoven pulls the full dependency stack — play (Pillow), video (MoviePy / imageio-ffmpeg), cloud clients (OpenAI + Anthropic), and local SLM (torch / transformers / accelerate / huggingface_hub):

pip install beatoven

pip install -U beatoven
beatoven --help
python -c "from beatoven import compose; print(compose('Type Hi', provider='heuristic').summary())"

pip install "beatoven[dev]"          # pytest, ruff, build, twine
pip install "beatoven[mcp]"          # MCP stdio server for Cursor / Claude Code
pip install "beatoven[diffusers]"    # local HF image-guide tiers (SD / SDXL)
pip install "beatoven[camera]"       # OpenCV webcam record
pip install "beatoven[record]" "beatoven[whisper]"  # mic + STT

Package: https://pypi.org/project/beatoven/

From source (development)

git clone https://github.com/ehallford11714/beatoven.git
cd beatoven
python -m venv .venv
.\.venv\Scripts\pip install -e ".[dev]"
python -m pytest -q

Quick start

from beatoven import compose

anim = compose(
    "Type 'Beatoven' letter-by-letter, then drop a rubber ball in a Three.js room",
    languages=["javascript", "python"],
    provider="heuristic",  # or auto / local / openai / claude / grok
)

print(anim.plan.summary())
print(anim.code["javascript"][:400])

anim.play()                       # auto-preview from generated languages
anim.play(playback="web")         # HTML from javascript
anim.available_playbacks()        # what's previewable

anim.edit("make the title blue and stagger faster")
anim.edit(path="scene/act1/shot1/title", props={"text": "Beatoven Live"})
anim.export("out/index.html")

Image guide (HF local → physical grounding)

anim = compose(
    "Rubber ball on a teal floor, warm key light",
    image_guide=True,            # or {"force_heuristic": True}
    image_guide_deepen=2,
    provider="heuristic",
)
anim.deepen_guide(levels=1)
print(anim.artifacts["image_guide"]["image"]["path"])

beatoven image-guide ladder
beatoven image-guide generate --prompt "glass sphere on concrete" --heuristic
beatoven compose --prompt "bounce" --image-guide --image-guide-deepen 2

Guide: docs/IMAGE_GUIDE.md.

Max engine suites (Unity / Unreal / OSS)

anim = compose(
    "Cinematic bounce with Timeline and Cinemachine",
    max_feature=True,            # or max=True
    engine="unity",              # unity | unreal | oss | all
    provider="heuristic",
)
anim.export_project("out/unity_max", target="unity")
anim.play(playback="unity")      # IR preview + project scaffold

beatoven catalog --engines
beatoven compose --prompt "Unreal Lumen glass" --max --engine unreal --lang csharp,python

Guide: docs/ENGINE_SUITES.md.

Draft + edit

anim = compose("Type 'Draft Scene' + bouncing ball", draft=True)
anim.edit("use Typed.js style typing; title color cyan")
anim.finalize()
anim.play()

Record suite (audio → STT library → typed playout)

Dedicated beatoven.record suite records microphone audio, transcribes with a chosen library (whisper / faster-whisper / openai / heuristic), then plays the text out via typed.js (or gsap/anime/motion):

from beatoven.record import RecordSuite, capture_and_play

capture_and_play(force_text="Hello from voice", playout="typed", out="out/from_voice.html")

suite = RecordSuite(transcriber="whisper", playout="typed")
suite.record(seconds=4)
suite.transcribe()
suite.playout(out="out/typed.html")

pip install "beatoven[record]" "beatoven[whisper]"
beatoven record capture --force-text "Demo" --playout typed --out out/from_voice.html
beatoven record libraries

Guide: docs/RECORD.md.

Video import / record + text overlay

Native beatoven.video module overlays typed text animations onto imported or recorded footage:

from beatoven import compose
from beatoven.video import VideoStudio

studio = VideoStudio.import_file("clip.mp4")
studio.overlay_text("Hello", out="out/hello.mp4", position="lower-third")

anim = compose("Type 'Live' + bouncing ball", provider="heuristic", export=False)
studio.overlay_animation(anim, out="out/live.mp4")

# Or in one compose call:
compose("Type 'Caption'", provider="heuristic", video="clip.mp4", out="out/caption.mp4")
# Webcam (needs: pip install beatoven[camera]):
compose("Type 'Rec'", provider="heuristic", record={"seconds": 4}, out="out/rec.mp4")

pip install "beatoven[camera]"   # opencv for webcam record
beatoven video overlay clip.mp4 --prompt "Type Hi" --out out/hi.mp4
beatoven video record --seconds 5 --out out/rec.mp4

Full guide: docs/VIDEO.md.

Output file types (gif / mp3 / html / …)

Pass the animation file type into compose — it sets plan.outputs and can write the file immediately:

anim = compose(
    "Type 'Beatoven' then bounce a ball",
    provider="heuristic",
    output="gif",                 # or filetype= / format=
    out="out/scene.gif",          # writes the file
)
print(anim.plan.outputs)          # ['gif']
print(anim.artifacts["export"])   # {'format': 'gif', 'path': ...}

# also: mp3, wav, html, mp4, png, jpg, svg, json, python, javascript, …
anim = compose("Type 'Hi' as html", provider="heuristic", filetype="html", out="out/index.html")

CLI:

beatoven compose --prompt "Type Hi" --output gif --export-file out/scene.gif
beatoven compose --prompt "Type Hi" --filetype mp3 --out out
beatoven export out/beat.json --out out/clip.gif --format gif

Grounded iterative satisfaction

Compose now verifies the plan/code against the prompt using a documentation contract (libraries, physics, engines, typing) and iteratively repairs gaps until the threshold is met (default on):

anim = compose(
    "Type 'Nova' then drop a rubber ball in a Babylon physics room",
    provider="heuristic",
    grounded=True,              # default
    max_ground_iters=3,
    satisfaction_threshold=0.85,
)
print(anim.verify().summary())  # [PASS] score=...
anim.ground(max_iters=2)        # force another grounded repair loop

Playback (anim.play(playback=…))

Specify preview on play(), not at compose time. Default playback="auto" picks the best match for whatever was generated:

anim = compose("Type 'Beatoven' then bounce a ball", provider="heuristic")

anim.play()                              # auto from generated code
anim.play(playback="python")             # pygame/tk IR sampler
anim.play(playback="web", object_playback=False)  # HTML file
anim.play(playback="unity")              # project scaffold + IR preview
print(anim.available_playbacks())

js = anim.to_js_object(show_when_done=True)

Deep Mode + multicascade

Deep Mode expands capability coverage. Cascade runs multi-pass SLM/heuristic refinement (structure → physics → cinematography → detail → engine bind) for Babylon / Unity / Bevy / Three complex scenes:

anim = compose(
    "Babylon physics room with stacked boxes and a bouncing ball",
    provider="auto",
    cascade=True,
    cascade_passes=4,
)
print(anim.plan.enhancements["cascade"])
anim.play(runtime="python", show_when_done=True)

anim.deep("more cinematic camera and bloom")
anim.cascade("richer multi-body physics", passes=3)

Agents (MCP / Cursor / Claude Code)

Coding agents can call Beatoven directly via MCP tools or the in-process AgentHook (no SDK required):

pip install "beatoven[mcp]"
beatoven agents install          # writes .cursor/mcp.json + .mcp.json + skills
beatoven agents status
python -m beatoven.mcp --list-tools
# or: beatoven-mcp / beatoven mcp

from beatoven.connective import AgentHook

hook = AgentHook()
r = hook.call_tool("beatoven_compose", {"prompt": "Type 'Hi'", "provider": "heuristic"})
assert r["ok"]
print(hook.call_tool("beatoven_verify", {}))

Full setup: docs/MCP.md.

CLI

python -m beatoven compose --prompt "Type 'Hello' then bounce a ball" --lang javascript,python --play
python -m beatoven compose --prompt "..." --deep --play
python -m beatoven compose --prompt "Babylon physics stack" --cascade --cascade-passes 4 --play
python -m beatoven compose --prompt "bounce" --image-guide --image-guide-deepen 2
python -m beatoven compose --prompt "Timeline title" --max --engine unity --lang csharp
python -m beatoven compose --prompt "..." --draft --out out
python -m beatoven edit out/beat.json --instruction "make the title blue and stagger faster"
python -m beatoven deep out/beat.json --instruction "cinematic bloom + dolly" --play
python -m beatoven cascade out/beat.json --passes 4 --play
python -m beatoven play out/beat.json --playback auto
python -m beatoven play out/beat.json --playback web --html
python -m beatoven export out/beat.json --out out/clip.html
python -m beatoven catalog --domain ray_tracing
python -m beatoven catalog --engines
python -m beatoven image-guide ladder
python -m beatoven probe --gpu --deep
python -m beatoven mcp --list-tools
python -m beatoven agents install

What's new in 0.12.0

• Image guide module — HF local / heuristic plates → physical grounding → hierarchical scaffold
• Max engine suites — Unity / Unreal / Godot / Stride / O3DE / Flax / Bevy full surfaces
• Extended catalog — pathtracer, PBR textures, Mitsuba/Blender, Taichi/MuJoCo, TripoSR/SF3D/Hunyuan
• anim.play(playback=…) — preview whatever language/engine was generated
• See docs/IMAGE_GUIDE.md, docs/ENGINE_SUITES.md, docs/DEEP_RENDER.md, CHANGELOG.md

Providers

Value	Behavior
auto	Cloud if API key present → else local Qwen when configured → else heuristic
openai / claude / grok	Cloud APIs
openai_compatible	BEATOVEN_LLM_BASE_URL + key + model
local	On-device Qwen/HF sized via hardware probe
heuristic	Offline catalog + hierarchy planner

Environment: OPENAI_API_KEY, ANTHROPIC_API_KEY, XAI_API_KEY / GROK_API_KEY, BEATOVEN_LLM_API_KEY, BEATOVEN_LLM_BASE_URL, BEATOVEN_LLM_MODEL, BEATOVEN_PROVIDER, BEATOVEN_LOCAL_MODEL.

Documentation

Doc	Description
docs/INDEX.md	Docs home
docs/TUTORIAL.md	Step-by-step tutorial
docs/API.md	Public API reference
docs/COMPONENTS.md	Every module explained
docs/CLI.md	CLI reference
docs/LIBRARIES.md	Capability pack catalog
docs/DEEP_RENDER.md	Textures, path tracing, HF on-device 3D
docs/ENGINE_SUITES.md	Max Unity / Unreal / OSS engine suites
docs/IMAGE_GUIDE.md	HF local image-guided anim/sim scaffolding
docs/ARCHITECTURE.md	Pipeline & Deep Mode
CHANGELOG.md	Release notes

Example prompts

See examples/prompts.md. Runnable scripts:

• examples/text_typing_web.py
• examples/draft_and_edit.py
• examples/threejs_physics_scene.py
• examples/deep_mode_demo.py

Layout

src/beatoven/
  capabilities/   # packs + full_surfaces + extended + engine_suites
  engines/        # max Unity/Unreal/OSS suite loader
  image_guide/    # HF local image → grounding → scaffold
  models/         # on-device neural-3D / texture ladders
  generators/     # JS / Python / Rust / C# / Java
  runtime/        # play + playback resolve + export
  providers/      # heuristic / local / cloud
  record/ video/ mcp/
tests/
examples/
docs/

License

MIT