you  →  "add a MIDI track called Bass, put Wavetable on it,
          write a rolling bassline in F minor, then close the filter a bit"

Claude →  creates the track · loads the instrument · writes 40 notes ·
          reads the filter's real range · sets cutoff · reports back "453 Hz"

What it does

48 tools in total. Full reference below.

How it works

Claude launches hellyee as a subprocess and talks to it over MCP. hellyee speaks OSC to AbletonOSC, a remote script running inside Live's own Python, which drives the Live Object Model.

Stock AbletonOSC exposes a lot, but not the browser, the master track, or arrangement clips. hellyee ships handlers that add all three, plus a patcher that installs them.

Install

1. Run the installer

If you have uv — no Python setup needed at all:

uvx hellyee setup

Otherwise:

pip install hellyee
hellyee setup

curl -LsSf https://astral.sh/uv/install.sh | sh     # macOS · Linux
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"   # Windows

hellyee setup downloads AbletonOSC, patches it with the browser / master / arrangement handlers, and writes your Claude config. It is idempotent — run it again any time. Use --client desktop for Claude Desktop, or --client both.

Want the audio features (hum-to-MIDI, voice commands)? They add ~380 MB, so they are opt-in:

pip install "hellyee[audio]"

2. Set up Live

This step is manual — Live has no API for enabling its own control surfaces.

1. Quit and reopen Live. Remote Scripts are only scanned at startup.
2. Open settings:
   • Live 12: Settings → Link, Tempo & MIDI
   • Live 11: Preferences → Link/Tempo/MIDI
   • Cmd + , on macOS · Ctrl + , on Windows
3. In the Control Surface table, pick AbletonOSC in the first free row.
4. Leave Input and Output as None — it communicates over the network, not MIDI ports.
5. You should see AbletonOSC: Listening for OSC on port 11000 in Live's status bar.

Once only. Live remembers it.

3. Check it

With Live open:

hellyee doctor     # connection, handlers, optional features
hellyee smoke      # full end-to-end test, cleans up after itself

smoke creates a real track, writes and quantizes notes, and loads an instrument, then deletes the track. Pass --keep to leave it in place.

Using it

Say what you want. Claude reads the set's state first, then acts.

"make a 4-bar house beat at 124 BPM"
"add a MIDI track called Bass and put Wavetable on it"
"write a rolling bassline in F minor, offbeat eighths"
"quantize that to 16ths at 0.7 strength so it still breathes"
"put an Auto Filter on the bass and close it down a bit"
"this lead is harsh — round off the highs and slow the attack"
"balance the mix, kick should sit on top"
"arrange this into a full track: intro, build, drop, breakdown, drop, outro"
"sweep the filter open across the last 8 bars before the drop"

Conventions worth knowing

Time is in beats. One 4/4 bar is 4 beats; a 16th note is 0.25.

Pitches use Live's display convention: C3 = 60. Standard MIDI notation calls that C4. hellyee follows Live so the note Claude writes matches the note you see.

Device parameters are set by percent, not by unit. Live's raw values live on internal scales that are not what the UI shows — Auto Filter's Frequency runs 20–135 but reads as "265 Hz". Set parameters with percent (0–100 across the parameter's own range); the tool reports back the displayed value so you can confirm what actually happened.

Tools

Audio input

Claude's API does not accept audio, so audio is processed locally and reaches the model as text or JSON:

notes_from_audio is monophonic only — humming, single-note lines. It will not transcribe chords or a full mix; use a polyphonic model such as basic-pitch for that.

On Apple Silicon, pip install mlx-whisper makes transcription much faster; hellyee prefers it when present. The first run downloads a model (~500 MB).

Known limitations

Claude cannot hear. It can measure output levels through Live's meters and reason about frequency ranges, but it cannot judge tone. EQ and sound-design choices come from convention and measurement — the final call is your ears.

Third-party plugins are opaque. Live does not expose VST/AU parameters to the API until you expose them by hand. Serum, Vital and friends will load and play, but Claude sees one parameter: Device On. To unlock a plugin, hit Configure on its device header, click the knobs you want controllable, then exit Configure — those parameters then appear.

Metering runs at ~10 Hz. AbletonOSC processes on a 100 ms tick, so meters measure sustained level, not transient peaks.

Quantize is client-side. Notes are read, snapped in Python, written back. That is why strength exists — but it costs a round trip rather than being instant.

Automation must start in a session clip. Live only creates envelopes on session clips, so hellyee writes automation there and carries it into the arrangement when the clip is placed. To vary automation across sections, write several clip variants and place the right one in each.

Session clips override the arrangement. If a track has ever had a session clip fired, it ignores arrangement clips until Back to Arrangement is pressed. hellyee handles this, but it is worth knowing when something plays silently.

No undo grouping. Each operation is its own step in Live's undo history.

Development

hellyee/
  osc.py            OSC client — persistent socket, request/response matching
  core.py           Live operations as plain functions (no Claude dependency)
  music.py          scales, chords, quantization, key-aware spelling
  audio.py          audio → notes, speech → text
  mcp_server.py     MCP tool layer
  cli.py            connection tests and diagnostics
abletonosc_patch/
  browser.py        adds browser access to AbletonOSC
  master.py         adds master track + arrangement to AbletonOSC
  setup_cli.py      installer: download, patch, configure Claude
abletonosc_patch/
  → shipped inside the wheel as hellyee/_patch

Working on hellyee itself:

git clone https://github.com/guvense/hellyee.git && cd hellyee
uv sync --extra audio          # or: pip install -e ".[audio]"
hellyee setup                  # re-applies the patch from your working copy

core.py holds the logic and knows nothing about Claude, so it is testable on its own and drivable from any front end. mcp_server.py is a thin layer of tool definitions over it.

⚠️ Editing anything in abletonosc_patch/? Those files run inside Live, which embeds Python 3.7. Walrus operators (:=), builtin generics (list[str]) and X | Y unions will not parse. The patcher does not check this for you — but python -c "import ast; ast.parse(open('file').read(), feature_version=(3,7))" does.

Hot reload. Editing an already-loaded handler does not need a Live restart — send /live/api/reload and it picks up the change in seconds. Adding a new module does require a restart, which is why browser and master handlers each live in one file.

Contributing

Issues and pull requests welcome. Useful directions:

• Polyphonic audio-to-MIDI (basic-pitch)
• Return tracks and sends
• Windows testing (developed on macOS)
• A skill layer with genre conventions and arrangement templates

Credits

Built on AbletonOSC by Daniel Jones, which does the hard work of exposing Live's Object Model over OSC.

License

MIT

hellyee