nabla-cli 0.9.2

Call-site harvester: scan OpenAI call sites, record real traffic, emit site profiles

nabla — OpenAI call-site harvester

nabla finds the OpenAI call sites in a Python repo, records real (prompt, completion) traffic for one of them, and emits a single self-contained site profile (site_profile.json) describing that call site: prompt template + slots, resolved output JSON Schema, sampling parameters, recorded examples, an inferred goal with difficulty dimensions, and a cost/latency baseline.

The profile is the handoff artifact for training a small, cheap replacement model for that one call site — everything downstream codes against the profile, never against your repo.

Install

pip install nabla-cli
# or, isolated:
pipx install nabla-cli

Requires Python ≥ 3.11.

nabla not found after pip install?

That means pip's script directory isn't on your PATH (pip prints a yellow warning about this during install). Two fixes, either works:

• Run it as a module instead — always works, no PATH needed:

  python -m nabla scan path/to/repo
  

• Or add pip's script directory to PATH. Find it with python -c "import sysconfig; print(sysconfig.get_path('scripts'))" and add that folder to your PATH (on Windows: Settings → search "environment variables" → edit Path → add the folder, then open a new terminal).

pipx install nabla-cli avoids the problem entirely if you have pipx.

Quickstart

Run from inside the target repo:

pip install nabla-cli

nabla init         # one-time setup, once per machine
nabla scan
nabla capture
nabla build

No flags needed on the happy path: the repo defaults to the current directory, capture writes <site>.captured.jsonl per site, and build finds those files automatically by the same convention.

With one supported call site, capture and build pick it automatically. With several, capture opens a multi-select (space toggles a site, a selects all, enter runs) and captures the selected sites in parallel, one progress bar each; build then produces a profile for every site that has captured pairs. --site <symbol> narrows any command to one site.

If you don't pass --inputs, capture looks for <site>.inputs.jsonl and otherwise generates a handful of synthetic inputs from the site's prompt template (saving them to that file so you can edit them and rerun with realistic data). Profiles built on generated inputs are marked generated_samples in their provenance — fine for a first look, but prefer real sample data for a profile you intend to hand off.

nabla init

Interactive, run once per machine:

$ nabla init
Pick a provider:
  1) openai
  2) groq
  3) gemini
> 2
Paste your GROQ_API_KEY: ****************
Saved to ~/.nabla/config.json

That's the only credential nabla needs. Base URLs, models, and request pacing all come from a built-in preset for whichever provider you picked — see Providers.

nabla scan

SYMBOL             FILE                     KIND   VERIFIABILITY  FLAGS
extract_invoice     app/invoice_extract.py  create json_schema    -

1 call site found.

nabla capture

One progress line per sample, then a summary:

nabla capture: sample 1/6 -> 1 capture(s)
nabla capture: sample 2/6 -> 1 capture(s)
...
captured 6 (prompt, completion) pairs from site extract_invoice -> extract_invoice.captured.jsonl

--out defaults to <site>.captured.jsonl in the current directory (so build can find it by the same convention afterward) — pass --out yourself only to pick a different name or location.

To record your own inputs instead of generated ones, pass --inputs (or create <site>.inputs.jsonl, which is picked up automatically): a JSONL file, one line per call to record. Each line is {"input_slots": {...}}, where the keys match the site function's keyword arguments (as reported by scan):

{"input_slots": {"document_text": "Invoice INV-1001 total $250.00 due 2026-08-01", "locale_hint": "en-US"}}
{"input_slots": {"document_text": "Facture FR-552 montant 99,50 EUR", "locale_hint": "fr-FR"}}

nabla runs the site's function once per line, in a subprocess, with its OpenAI client rerouted through a local recording proxy.

nabla build

nabla build: goal — "Extract structured invoice fields (total, currency,
due date) from free-form invoice or receipt text, normalizing
locale-specific number and date formats."
nabla build: wrote extract_invoice.profile.json (site 4f2a9b1c3d8e…, supported=True, 6 examples)

--out defaults to <site>.profile.json in the current directory. --captured is optional: build looks for <site>.captured.jsonl in the current directory — the file capture just wrote — and you only need --captured <path> to point it at a different recording. The one LLM call in this step is goal inference: it reads the reconstructed prompt template, the resolved output schema, and the recorded examples, and returns a goal description, implicit constraints, and difficulty dimensions with cited evidence. That's the goal line printed above, and it's what lands in the profile's goal block.

nabla jeremy

An alternative to build with the same inputs and site handling: instead of the full site profile, it emits <site>.jeremy.json — a seed spec for a problem generator:

{
  "goal": "<build's inferred goal description and constraints, joined into one string>",
  "system_prompt": "<the site's own system prompt>",
  "examples": [ {"problem": "...", "solution": "..."}, ... ]
}

Same pipeline as build (captured pairs in, one goal-inference call) — only the output shape differs; nothing is added that build's own output doesn't contain.

nabla push

Hands built artifacts to the backend dispatcher, which runs the seed-generation agent pool against them: every <site>.profile.json and <site>.jeremy.json found for the repo's supported sites (narrow with --site, or push exact files with --artifact). Artifacts are translated into the dispatcher's contract — {goal, examples, student_system_prompt, student_goal} — where student_system_prompt is the site's own production system prompt. A push runs the full pool server-side and may take minutes.

The endpoint defaults to the local dispatcher (http://127.0.0.1:8000/run-task; start it from backend/ with langgraph dev and uvicorn seed_generator.server:app --port 8000) and is meant to change later — resolution order: --endpoint flag, then NABLA_PUSH_ENDPOINT, then push_endpoint in ~/.nabla/config.json, then the default.

nabla pull (preview)

The loop-closer: fetches ready fine-tuned deployments and rewrites each harvested call site to run on your model — client base_url + model swap with env-wired credentials, rollback via one unset. Dry-run by default (diff only); --apply performs the edits and verifies the site live against the new endpoint, then prints the before/after cost and latency.

Currently a full UX simulation while the deployments API is built: every run renders the real intended flow (diffs come from your repo's actual source) but writes nothing — each run ends with an explicit "simulation — nothing was written" line.

Providers

nabla init supports three providers. Pick whichever you have a key for — it doesn't need to be the provider your target repo already calls.

Provider	Default model	Role
openai	your repo's own model (goal inference uses gpt-4o-mini)	Real upstream — capture calls OpenAI directly with your key, auth passthrough.
groq	openai/gpt-oss-120b	Stand-in oracle — capture calls Groq instead of OpenAI.
gemini	gemini-3.5-flash	Stand-in oracle — capture calls Gemini instead of OpenAI.

When you pick groq or gemini, they only stand in for generating the completions during capture. The profile's cost/latency baseline always reports numbers for the target repo's own model — the one your code actually calls in production — never the stand-in's.

Gemini's free tier has a very low daily quota: expect capture to pace requests with multi-second delays and to still hit rate limits past a handful of samples. groq or openai handle larger sample sets more comfortably.

What gets detected

scan classifies every chat.completions.create / beta...parse call by verifiability: json_schema and pydantic_parse sites are supported harvest targets; tool_call, json_object_no_schema, and free_text sites are detected and reported but not harvested. Streaming, multi-turn, vision, and n>1 sites are flagged unsupported. Generic wrapper functions are expanded to their callers; functions that own their prompt template are kept as the site.

Advanced: overrides

Flags

Auto-detection is a default, not a requirement — the explicit flags still work and take priority over it:

• --site <symbol> — skip site auto-pick; required once a repo has more than one supported call site.
• --out <path> (capture) — write recorded examples somewhere other than <site>.captured.jsonl.
• --captured <path> (build) — read recorded examples from somewhere other than <site>.captured.jsonl.
• --out <path> (build) — write the profile somewhere other than <site>.profile.json.

Environment variables

Everything nabla init sets up can also be overridden by hand. Precedence, highest first:

1. An env var you set yourself
2. ~/.nabla/config.json (written by nabla init)
3. The provider preset's built-in defaults

Variable	Purpose
OPENAI_API_KEY	Default upstream for capture (auth passthrough).
GROQ_API_KEY / GEMINI_API_KEY	That provider's conventional key env, read if ~/.nabla/config.json has no stored key.
NABLA_PROVIDER	Overrides the provider nabla init selected.
NABLA_UPSTREAM_BASE_URL	Record through any OpenAI-compatible stand-in oracle instead.
NABLA_UPSTREAM_API_KEY	Key for the stand-in oracle.
NABLA_UPSTREAM_MODEL	Rewrite the model for the stand-in only (the target repo's own model still lands in the profile).
NABLA_RECORD_DELAY_MS	Pacing between samples for rate-limited upstreams (default 0).
NABLA_SAMPLE_TIMEOUT	Per-sample subprocess timeout in seconds (default 300).
NABLA_INDUCE_BASE_URL / NABLA_INDUCE_API_KEY_ENV / NABLA_INDUCE_MODEL	Swap the goal-inference provider/model.

Notes

• capture runs the site's enclosing function in a subprocess per sample, with OPENAI_BASE_URL pointed at a local recording proxy. If the target's tests mock the SDK, the proxy path reports zero traffic explicitly instead of writing an empty profile.
• Recorded prompts/completions are real data from your repo and leave the machine only via the goal-inference call. Review before sharing profiles.