sceneprogagent 0.1.9

Autonomous, DSL-fluent coding agent built on the Claude Agent SDK — point it at any DSL directory and it plans, codes, debugs, verifies, and self-extends.

sceneprogagent

An autonomous, DSL-fluent coding agent built on the Claude Agent SDK.

Point it at a DSL directory — a folder of .py/.md/.html files describing your domain API. The agent plans, retrieves the right DSL symbols, writes code, runs it the way your DSL says to, debugs until it runs cleanly (and, optionally, until the result is verified correct), and can autonomously extend the DSL when a task needs a capability it lacks.

It is domain-agnostic: 3D generation, Blender, SfM, scientific simulation, or any other domain is just a different DSL directory. No domain assumptions live in this package.

---

Prerequisites

Requirement	Notes
Python ≥ 3.10
Claude Code CLI	npm install -g @anthropic-ai/claude-code (or download from claude.ai/code)
Authentication — see below
git	Only needed for autonomous DSL self-extension (extension_enabled=True)

Authentication

The agent drives the claude CLI as a subprocess — auth is handled by that process. Pick whichever method fits your setup:

Option A — Anthropic API key (pay-per-token, from console.anthropic.com)

# shell (recommended for scripts / CI)
export ANTHROPIC_API_KEY=sk-ant-...

Or pass it programmatically — no shell env var needed:

agent = SceneProgAgent(AgentConfig(dsl_dir="./dsl", api_key="sk-ant-..."))

Or via the CLI flag:

sceneprogagent run --dsl ./dsl --api-key sk-ant-... "My task"

Option B — Claude.ai / Claude Code login (Claude Pro/Max/Teams subscription)

claude auth login    # one-time; stores credentials in ~/.claude/.credentials.json

After that, no API key is needed — the subprocess reads the stored credentials automatically.

Running as root (Docker)

AgentConfig automatically sets IS_SANDBOX=1 for the CLI subprocess when permission_mode="bypassPermissions" (the default). You do not need to set it manually.

If the CLI is not on PATH, tell the agent where to find it:

export CLAUDE_CODE_EXECPATH=/path/to/claude   # or pass cli_path= to AgentConfig

---

Install

From PyPI (recommended)

pip install sceneprogagent

With optional embedding-based doc search:

pip install "sceneprogagent[embed]"

From source

git clone https://github.com/KunalMGupta/sceneprogagent
cd sceneprogagent
pip install -e .              # core
pip install -e ".[embed]"     # + chromadb / sentence-transformers
pip install -e ".[dev]"       # + pytest

---

Quick start (5 minutes)

1 — Point at your DSL directory

Your DSL directory needs at minimum one .py file with the API and a runbook — a markdown file telling the agent how to execute scripts with this DSL.

my_project/
├── dsl/
│   ├── api.py          ← your domain Python library
│   ├── RUNBOOK.md      ← execution recipe (see below)
│   └── docs.md         ← optional extra docs
└── solve.py            ← your script using sceneprogagent

dsl/RUNBOOK.md (minimal):

# My DSL

Import the API with `import api` and call its functions.

## How to run code
```run
python3 {file}
```

The ```run block is the execution recipe. {file} is replaced with the path to each generated script. For a Blender DSL it might be blender --background --python {file}; for SfM it might be python3 {file} with a special env.

2 — Run from the CLI

# one-shot task
sceneprogagent run --dsl ./dsl "Create a red cube and save it to output.blend"

# save the final verified program
sceneprogagent run --dsl ./dsl --save-code final.py "Create a red cube"

# iterative optimization
sceneprogagent run --dsl ./dsl --optimize "minimise polygon count" "Create a sphere"

# disable autonomous DSL self-extension
sceneprogagent run --dsl ./dsl --no-extend "Create a box"

3 — Use from Python

The one-call interface — description in, program + produced files out (a filled Pydantic result):

from sceneprogagent import generate

r = generate("Create a simple brick material", dsl="./dsl", out_dir="results/")
r.code        # the executable program (str)
r.artifacts   # produced files, copied to results/  e.g. ['results/clay.blend', 'results/render0.png']
r.summary     # what the agent built

Want specific fields? Pass your own Pydantic model and the agent fills it:

from pydantic import BaseModel, Field
from sceneprogagent import generate

class MaterialOutput(BaseModel):
    code: str = Field(description="The executable DSL program.")
    blend_file: str = Field(description="Path to the saved .blend file.")
    renders: list[str] = Field(default_factory=list, description="Rendered preview images.")

out = generate("Create a glossy copper material and render it", dsl="./dsl",
               result_model=MaterialOutput)
out.blend_file, out.renders

Or drive the agent directly when you need more control (reuse across tasks, async, etc.):

from sceneprogagent import AgentConfig, SceneProgAgent

with SceneProgAgent(AgentConfig(dsl_dir="./dsl")) as agent:
    summary = agent.run("Create a scene with a box named 'table' of size 2 and save it")
    print(agent.final_code)   # the final program

---

Deploying in a new project

Step 1 — Install

pip install sceneprogagent

Step 2 — Create a DSL directory

A DSL directory needs:

• At least one .py file with your domain API (public functions, well-named, with docstrings).
• A runbook named RUNBOOK.md (also discovered as AGENT.md, DSL.md, or README.md).

The runbook teaches the agent two things: (1) how to execute scripts, via the ```run block; (2) any domain conventions and gotchas, in plain markdown prose.

Example structure:

# My Domain DSL

Use `import myapi` to access all functions.

All functions work with absolute paths. Pass `save=True` to persist results.

## How to run code
```run
python3 {file}
```

## Tips
- Always call `myapi.init()` first.
- `myapi.save(path)` requires the parent directory to already exist.

Step 3 — Configure and run

from sceneprogagent import AgentConfig, SceneProgAgent

config = AgentConfig(
    dsl_dir="./dsl",             # path to your DSL directory
    work_dir="./agent_runs",     # scratch dir for code attempts + logs (auto-created)
    model="claude-sonnet-4-6",   # any Claude model id (Opus for the hardest tasks)
    max_turns=80,                # agentic tool round-trips per run
    max_debug_iters=8,           # debug retries before giving up
    max_budget_usd=1.00,         # optional spend ceiling
    extension_enabled=True,      # let the agent extend the DSL when needed
    save_code="out/final.py",    # write the final program here
)

with SceneProgAgent(config) as agent:
    summary = agent.run("Your task description here")
    print(summary)
    print(agent.final_code)

---

Python API reference

AgentConfig

Parameter	Default	Description
dsl_dir	(required)	Path to the DSL directory
work_dir	<dsl>/../.sceneprog_work	Scratch dir for code attempts, logs
runbook_path	auto-discovered	Override runbook location
api_key	None	Anthropic API key (sk-ant-...). Falls back to ANTHROPIC_API_KEY env var or stored claude auth login credentials
model	"claude-sonnet-4-6"	Claude model ID (use "claude-opus-4-8" for hard tasks)
max_turns	80	Max tool round-trips per run()
max_debug_iters	8	Max debug retries before giving up
max_budget_usd	None	Optional spend ceiling per run
extension_enabled	True	Allow autonomous DSL self-extension
embed_enabled	False	Use chromadb embeddings for doc search
verify	False	Require the agent to confirm correctness with verify_solution before finishing. Off by default — the agent stops once code runs cleanly (cheaper). Set True for a correctness gate
save_code	None	Default path to write the final program
cli_path	auto-discovered	Override path to the claude CLI binary
permission_mode	"bypassPermissions"	Claude SDK permission mode
extra_env	{}	Extra env vars for the subprocess

generate (one-call interface)

generate(
    description: str,
    dsl: str | Path,
    *,
    result_model: type[BaseModel] = GenerationResult,  # your own model works too
    out_dir: str | Path | None = None,                 # copy produced files here
    model: str = "claude-sonnet-4-6",
    verify: bool = False,
    extension_enabled: bool = True,
    work_dir: str | Path | None = None,                # explicit -> persistent
    keep_workdir: bool = False,                         # keep the scratch dir
    **config_kwargs,        # forwarded to AgentConfig (max_turns, api_key, ...)
) -> BaseModel              # an instance of result_model, filled by the agent

GenerationResult (the default model) has code: str, artifacts: list[str], summary: str, work_dir: str. The same thing is available on an existing agent via agent.run_structured(task, result_model, out_dir=None) (and run_structured_async).

Scratch isolation. With no work_dir, each call does its intermediate work (code attempts, index, logs) in a disposable temp directory that is deleted when the call returns — so repeated/parallel calls don't clutter your filesystem. The produced files are copied to out_dir first (defaulting to ./sceneprog_output) so the paths in the returned model stay valid. Use keep_workdir=True to keep the scratch dir for debugging, or pass an explicit work_dir=... for a persistent location (never deleted).

SceneProgAgent

agent = SceneProgAgent(config)

# Solve one task (synchronous)
summary: str = agent.run(task, save_code=None)

# Iterative optimization
summary: str = agent.optimize(task, objective, max_rounds=3, save_code=None)

# Async variant (use inside an existing event loop, e.g. FastAPI, Jupyter)
summary: str = await agent.run_async(task, save_code=None)

# Result of the last run
agent.final_code    # str | None — the final verified program
agent.retriever     # query the DSL index without a model call

# Lifecycle
agent.close()       # close the trajectory log when done
# or use as a context manager:
with SceneProgAgent(config) as agent:
    ...

Init once, run many

A single SceneProgAgent is reusable — initialize once and solve tasks sequentially. Each call to run() resets per-task result tracking; the trajectory log spans the whole session.

with SceneProgAgent(AgentConfig(dsl_dir="./dsl")) as agent:
    for i, task in enumerate(my_tasks):
        summary = agent.run(task, save_code=f"out/task_{i}.py")
        results[i] = agent.final_code

---

How it works

task ─▶ plan ─▶ retrieve DSL context ─▶ write code ─▶ run via runbook ─▶ verify
                      ▲                                     │              │
                      └──────────── debug loop ◀────────────┘   fail ◀─────┘
                                                                  │ (capability missing)
                                                         autonomous DSL extension

• Hybrid retrieval grounds every call in the real API — no hallucinated functions:
  • dsl_overview — compact cheatsheet of all public symbols (always in context);
  • dsl_lookup — real signatures, docstrings, and source for any symbol;
  • dsl_search_docs + built-in Grep/Read for depth (optional embeddings via [embed]).
• Verification oracle (verify_solution, opt-in via verify=True): clean exit + an assertion script the agent writes, so "success" means the answer is correct, not just non-crashing. Off by default — see Cost & performance.
• Autonomous self-extension: git snapshot → WebSearch/clone → translator subagent adapts external code into DSL-style files → reindex → smoke-test → use. restore_dsl rolls back a bad extension.
• Trajectory log — every tool call, code submission, and verification result is written to <work_dir>/trajectory.jsonl; every generated script is kept in <work_dir>/code/attempt_NNN.py.

---

What the agent can and can't guarantee

Will reliably do:

• Read and use your DSL's actual API (not hallucinate functions)
• Run the generated code and show you the output
• Retry with the real error/traceback until it passes its own assertions
• Report final_code from the last program that ran cleanly (and, with verify=True, passed verification)

May still fail when:

• The task requires DSL capabilities that don't exist (use extension_enabled=True to let the agent add them)
• The agent misunderstands the task and asserts the wrong thing — phrasing matters; "verify it equals 17" beats "check it's roughly right"
• max_debug_iters is reached without a solution — increase it or break the task into smaller pieces

---

Advanced: querying the DSL index directly

agent = SceneProgAgent(config)

# No model call — instant, free
print(agent.retriever.cheatsheet())         # all public symbols
print(agent.retriever.lookup("my_func"))    # signature + source
print(agent.retriever.search_docs("save"))  # heading / prose search

This is useful for building your own tooling on top of the DSL index — e.g. autocomplete, a REPL helper, or a CI check that the DSL API hasn't changed.

---

Cost & performance

Most per-run cost comes from how much context is sent on every agentic turn. Three levers, biggest first:

1. Keep the DSL directory scoped to DSL source. The agent builds an API cheatsheet from every .py file under dsl_dir and keeps it in context on every turn. If the directory also contains a vendored runtime (a bundled interpreter, a site-packages, a whole app distribution), the cheatsheet fills with thousands of junk symbols — you pay for them every turn and the agent can't see the real API, so it burns extra turns rediscovering it. This is usually the dominant cost.

• Vendored interpreter trees (lib/python3.x, site-packages, dist-packages) are auto-skipped.

• For anything else, add a .sceneprogignore (gitignore-style) at the DSL root:

  # dsl/.sceneprogignore
  blender-4.5.4-linux-x64/   # bundled app/runtime — not the DSL API
  __pycache__/
  *.generated.py
  

  Check the effect with no model call:

  agent = SceneProgAgent(config)
  print(agent.retriever.cheatsheet())   # should show only your real DSL files
  

2. Model choice. The default is claude-sonnet-4-6 — strong at DSL codegen and ~5× cheaper than Opus. Use model="claude-opus-4-8" only for the hardest tasks.

3. Keep the runbook lean. The full runbook text is injected on every turn (it rides prompt-caching, but still counts). A very large worked example can be moved into a separate doc the agent pulls on demand via dsl_search_docs/Read instead of living in the always-injected runbook.

4. Verification is off by default. The agent stops as soon as the code runs cleanly. Turning it on (verify=True / --verify) makes the agent author a correctness check-script, run it, and often render+read output — valuable when correctness matters, but it roughly doubles the work. Leave it off when you just want executable code.

---

Isolation: running in a container

The agent is a capable computer-use agent (it runs bash, writes files). Two layers of isolation:

• Intermediate files are already handled in-library: generate() does its scratch work in a disposable temp dir that's deleted afterward (see Scratch isolation above), so your filesystem doesn't accumulate per-run junk.
• Host / environment isolation is a deployment concern, not the library's job. The clean pattern is to run the whole agent inside a container you control, with the DSL mounted read-only and an outputs directory bind-mounted. The library does not spawn or manage containers itself (no Docker-in-Docker).

A generate CLI subcommand makes this a one-liner entrypoint — it prints the structured result as JSON and copies produced files to --out:

# Dockerfile (built once, with your DSL's runtime deps, e.g. Blender + sceneprogexec)
FROM your-dsl-base:latest
RUN pip install sceneprogagent
# DSL is mounted at runtime (read-only); outputs bind-mounted
ENTRYPOINT ["sceneprogagent", "generate", "--dsl", "/dsl", "--out", "/out"]

docker run --rm \
  -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
  -v "$PWD/blender:/dsl:ro" \
  -v "$PWD/results:/out" \
  your-dsl-image "Create a simple brick material" \
  > result.json          # the filled Pydantic result; artifacts land in ./results

--rm tears the container down on exit; ./results keeps the produced files, and result.json holds the structured result (code + artifact paths). For programmatic use, call generate(...) from a Python entrypoint instead and read the returned model.

---

Module layout

Module	Responsibility
config.py	AgentConfig: paths, model/budget, toggles, CLI + sandbox env
runbook.py	discover/parse the runbook and its execution recipe
index/	static (AST) API index + cheatsheet; optional embedding store
retrieval.py	hybrid retriever (overview / lookup / docs)
execution.py	run generated code via the runbook recipe
verify.py	layered verification oracle (exit code + assertions + optional LLM judge)
extension.py	git snapshot/restore + repo cloning primitives
tools.py	the sceneprog MCP tool server (all agent-callable tools)
prompts.py	system prompt + debugger/translator subagent definitions
agent.py	main orchestrator — assembles options, drives the SDK loop
optimize.py	iterative-optimization loop
telemetry.py	JSONL trajectory logging
cli.py	command-line entry point

---

Tests

pytest                              # offline: index, retrieval, runbook, execution, verify
SCENEPROG_RUN_E2E=1 pytest -s       # + live agent tests (use real Claude; cost money/time)

---

Building and publishing to PyPI

pip install build twine

# build the distribution
python -m build                     # produces dist/sceneprogagent-*.whl and .tar.gz

# upload to TestPyPI first
twine upload --repository testpypi dist/*
pip install --index-url https://test.pypi.org/simple/ sceneprogagent

# when satisfied, publish to PyPI
twine upload dist/*

To bump the version, edit version = "..." in pyproject.toml and __version__ in sceneprogagent/__init__.py, then rebuild.

---

License

MIT © Kunal Gupta