foley

Find (or generate) the right sound effect for a moment of narration — and weave it in.

foley is a retrieval-first façade for sound effects: one simple surface over many sound sources (your own library, service APIs, and generative-AI models), a searchable index of every sound (by keyword and meaning), an agent that picks the right sound for a narrative context, and a compositor that places it under the voice.

It's the SFX sibling of arioso (a unified façade over AI music-generation backends): same discipline — one entry function, config-driven plugin adapters, a unified vocabulary translated per-backend, zero required core deps with lazy optional-deps — but centered on search rather than generation, with generation as just one of several sources.

Status: v1. All four stages — source, index, select, weave — plus the MCP server and the licensing/provenance, evaluation, and observability layers are implemented (Epic #13 complete). The API below is live; see the roadmap for what's next. Follow along in misc/docs/.

The idea

import foley

# The headline — right sounds for a narrative context:
# decompose the passage into salient sound events → search → verify → decide
candidates = foley.find("She pushed open the heavy oak door; rain hammered outside.")

# Direct hybrid search of your library (text query or a reference clip)
hits = foley.search("distant thunder rumble", k=10, commercial_ok=True)

# Generate a sound when nothing fits (arioso-style; pluggable backends)
clip = foley.generate("a single wooden door creak", backend="stable_audio", duration=3)

# Grow the library — ingest auto-tags, captions, and embeds every file
foley.ingest("~/my_sounds/")
foley.add_from("freesound", query="ocean waves", license="cc0")

# Compose: place the sounds under the narration (find → plan → weave)
timeline = foley.plan(candidates)  # the editable sound-design plan
result = foley.weave("narration.wav", timeline)  # mastered mix + SDH captions + credits

How it works — four stages

The selection tools publish as an MCP server (via py2mcp) so the same capabilities drive the agent, a CLI, and external hosts.

Design & research

foley's design is grounded in five research reports (unified, cited):

• Design · Roadmap
• Research reports — sound sources, SFX generation, recognition/ tagging, embeddings/indexing, and the context-retrieval agent
• Deep-research prompt library — 12 ready-to-run prompts covering every dimension of the project

Install

pip install foley

(Optional per-capability extras — foley[freesound], foley[clap], foley[stable-audio], foley[index], … — are added as each subsystem lands.)

License

MIT