This release is a pre-release and may not be stable for production use.
JAZ
A Python framework for building and optimizing LLM-agents through intelligent, iterative execution.
Overview
JAZ provides a unified framework for creating agents that combine deterministic Python code with LLM reasoning. Agents execute in a REPL loop where the LLM generates code, observes results, and iterates until the task is complete.
Prerequisites
- Python ≥ 3.12 — 3.14+ is recommended for the full feature set. The t-string prompt
syntax (
invoke(task=t"Analyze the {data}")) and the interactivejazconsole need PEP 750 t-strings, hence Python ≥ 3.14. Everything else works on 3.12/3.13; the equivalentinvoke(task="Analyze the `data`", data=data)works on every supported version.
Installation
pip install jaz-lang
The distribution is named jaz-lang; the import name is jaz:
import jaz
Optional extras:
pip install "jaz-lang[tracing]" # OpenTelemetry tracing
pip install "jaz-lang[copilot]" # GitHub Copilot backend — no API key, billed to AI credits
To pin a specific release, or to install one that has not reached PyPI, use the distribution repo and a release tag:
pip install "jaz-lang @ git+https://github.com/jaz-lang/jaz.git@v0.2.0b1"
API Keys
JAZ reads provider API keys from the environment, under each provider's standard variable name. Set them via any of:
- Shell profile — add
export OPENAI_API_KEY=sk-...to~/.zshrcor~/.bashrc(simplest, available globally) .envfile — create a.envat the project root withOPENAI_API_KEY=sk-...per line, then load it with:- direnv — add
dotenvto an.envrcfile to auto-load oncd(brew install direnv)
- direnv — add
No API key at hand? The copilot backend (the copilot extra) bills a Copilot
subscription's GitHub AI credits instead. Authentication checks an explicit github_token
first, then COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN in the environment
(in that order), then your signed-in gh/copilot CLI. An exported token therefore
takes precedence over your CLI login:
import jaz
from jaz.llm import GitHubCopilotLLM
jaz.configure(llm=GitHubCopilotLLM(model="gpt-5-mini")) # ids are Copilot's own, unprefixed
For CLI authentication, run gh auth login, or start copilot and enter /login.
Like every backend other than LiteLLM it sits outside the stable API surface — see Custom & Local LLM Backends.
Quick Start
from jaz import invoke
from jaz.hooks import ReturnType
# Simple task
result = invoke(
ReturnType(int),
task="Calculate the factorial of 10",
)
# With custom inputs
result = invoke(
ReturnType(list),
task="Double each number in the list",
numbers=[1, 2, 3, 4, 5]
)
Later snippets assume the Quick Start imports (from jaz import invoke,
from jaz.hooks import ReturnType, import jaz) and show only what each adds.
Every input is a keyword argument — the prompt included. invoke()'s positional
arguments are hooks (ReturnType(int), a budget, …) and an optional ConfigOverride, so
invoke("Calculate the factorial of 10") is an error, not a shorthand.
The jaz console
Installing puts a jaz command on PATH (it needs Python ≥ 3.14 to run): a real Python
REPL — variables persist, _ holds the last result — with conversational sugar layered
on top. Run jaz, then enter these lines at its prompt:
> what is 2 + 2?
n: int <- how many days in a leap year?
?n
% switch to gpt-5-mini
import math
> asks the agent; n: int <- captures a typed result; ?n shows what the agent sees
for n; % asks the jaz helper to change settings, with confirmation. Plain Python
still runs. Enter the sigil lines without trailing # annotations: they become prompt
text, or can leave a ? expression waiting for more input.
A session with no model configured opens a setup wizard first: pick a provider and paste
its API key, or pick github copilot — no provider key, just the copilot extra and
GitHub authentication. The choice is validated with one live request before
anything is kept. Backend and model defaults are saved to ~/.jaz/settings.json;
provider API keys entered in the wizard go in ~/.jaz/credentials.json.
Flags configure one session without touching that default — jaz --backend copilot --model gpt-5-mini, jaz --max-cost 0.50, jaz --log run.log — and jaz -c "…" runs a single
prompt and exits. In-session, setup_llm() re-runs the wizard and
switch_model("openai/gpt-5") changes model (keeping the backend) after validating it.
Key Features
Recursive Agent Composition
Agents can invoke nested agents with separate budgets and automatic cost tracking:
from jaz.hooks import RecursionLimit, ReturnType
with RecursionLimit(max_depth=3): # optional cap; recursion is unbounded by default
result = invoke(
ReturnType(list),
task="""
For each item, use invoke to process it.
Combine and return the results.
""",
items=data,
)
Custom Tools
A tool is just a function you pass in. Give it as a keyword argument and it binds to that name in the agent's REPL; its signature and docstring render in the prompt, so the docstring is the description the agent reads:
def my_tool(x: int) -> int:
"""Double a number."""
return x * 2
result = invoke(ReturnType(int), task="Use my_tool on 5", my_tool=my_tool)
The agent sees `my_tool(x: int) -> int`: Double a number. — type hints and a
one-line docstring are what make a tool usable. Nothing is registered up front, so tools
sit alongside data in the same call:
def web_search(query: str) -> list[str]:
"""Search the web and return result snippets."""
...
result = invoke(
ReturnType(dict),
task="Look up each name and summarize what you find",
names=["ada", "grace"],
web_search=web_search,
)
To group related tools under one name, pass an object instead: it renders as its class
docstring with one bullet per public method, and the agent calls them as
utils.my_tool(...).
class Utils:
"""Utility helpers."""
def my_tool(self, x: int) -> int:
"""Double a number."""
return x * 2
result = invoke(ReturnType(int), task="Use utils.my_tool on 5", utils=Utils())
The agent sees:
`utils` [Utils]: Utility helpers.
- `my_tool(x: int) -> int`: Double a number.
Data and other values render as their str(), so define __str__ on a type to control what
the agent sees for it, or __repr__ for a value that sits inside a list or dict.
To make a tool propagate automatically to nested invoke calls, bind it with
jaz.scope instead of passing it per call:
from jaz import scope
with scope(my_tool=my_tool):
invoke(ReturnType(int), task="Use my_tool on 5") # nested invokes inherit `my_tool`
Hooks for Extensibility
Use hooks to add logging, tracing, workflow strategies, and more:
from jaz.hooks import BudgetPool, FileLogger
with FileLogger("agent.log"), BudgetPool(cost_budget=1.0):
result = invoke(ReturnType(str), task="Build a web scraper")
Budget Control
Pool budgets (LLM cost / call count, shared across the whole invoke/recursion
tree) are enforced by the opt-in BudgetPool; per-level execution limits
are opt-in hooks as well:
from jaz.hooks import BudgetPool, IterationLimit, RecursionLimit
# Pool budgets: tracked + enforced only while the hook is active.
with BudgetPool(cost_budget=0.50, calls_budget=100): # USD / call-count
result = invoke(ReturnType(str), task="Complex task")
# Per-level execution limits are hooks too:
with IterationLimit(max_iterations=20), RecursionLimit(max_depth=2):
result = invoke(ReturnType(str), task="Complex task")
Configuration
Override defaults globally or per-invoke:
Each group takes the configured component, whose constructor is that group's settings. Setting a group replaces it — a component states itself completely, so there is no partial update of one setting.
from jaz.llm import LiteLLM
from jaz.repl import PythonREPL
# Per-invoke override
with jaz.ConfigOverride(llm=LiteLLM(model="openai/gpt-5-mini", temperature=0.7, max_tokens=2000)):
result = invoke(ReturnType(str), task="Creative task")
# Global configuration
jaz.configure(llm=LiteLLM(model="openai/gpt-5-mini"), repl=PythonREPL(exec_timeout=60))
Custom & Local LLM Backends
JAZ talks to LLMs through a small BaseLLM layer. LiteLLM is the default — one backend that routes to every provider LiteLLM supports (OpenAI, Anthropic, Gemini, Bedrock, Vertex, …) — and GitHubCopilotLLM and SGLangLLM cover the two cases it does not serve well: Copilot billing through GitHub AI credits, and a self-hosted SGLang server addressed token-in/token-out. All three are part of the stable API. You can add your own backend without forking.
A config selects one backend: the BaseLLM you pass to llm=, defaulting to LiteLLM. Its constructor takes everything it needs — the model id (a LiteLLM route like openai/gpt-5-mini), the backend's own settings, and any per-request params. A config file can name a backend instead of constructing one, but the name lives in whatever reads the file: JAZ's own loaders (the eval harness, the console) each keep their own tag → class mapping, and core carries neither the mapping nor a name on the class. Because litellm is the default, a config need only name the model.
Local OpenAI-compatible server (Ollama, vLLM, LM Studio, llama.cpp, …) — no code, just config:
# route the litellm backend to a local OpenAI-compatible server
# (api_key is required by the openai/ route even for a local server — a dummy value is fine;
# it rides the query params into litellm.completion)
jaz.configure(
llm=LiteLLM(model="openai/llama3", api_base="http://localhost:11434/v1", api_key="dummy"),
)
A custom backend — subclass BaseLLM and pass an instance (a name is only needed if a config file must select it, and it is that file's reader that holds the name):
import os
from functools import cached_property
from jaz.llm import BaseLLM, LLMResponse, ModelInfo, RetryPolicy
class MyLLM(BaseLLM):
def __init__(self, model: str, base_url: str | None = None, retry_policy=RetryPolicy(), **query_params):
super().__init__(retry_policy=retry_policy) # the one thing the base holds
self._model = model # yours: the base has no `model`
self._base_url = base_url or os.environ["MYBACKEND_API_BASE"]
self._query_params = query_params # yours too; you put them on the call
def complete(self, messages):
# ... call self._base_url; raise jaz.exceptions.RateLimitError / AuthenticationError / ...
# on API errors so retry classification works.
wire = my_http_post(self._base_url, model=self._model, messages=messages, **self._query_params)
return LLMResponse(
content=wire["text"],
prompt_tokens=wire["usage"]["input"],
completion_tokens=wire["usage"]["output"],
cost_usd=None, # or price it yourself; see below
extra={},
)
@cached_property
def model_info(self):
return ModelInfo(name=self._model) # what you know; every field defaults to "unknown"
jaz.configure(llm=MyLLM(model="my-model", base_url="http://localhost:8000"))
One class owns the whole job: the API call, its parameters, and whatever accounting you choose to report. complete() receives the messages and returns an LLMResponse — that is the entire contract, and BaseLLM, LLMResponse, ModelInfo and RetryPolicy are the only names you need from jaz.llm. Nothing passes through the base: the model, the sampling parameters and the credentials are your own state, set in your __init__ and put on the request by your complete. The base holds retry alone, so a backend with a **query_params tail names retry_policy explicitly and forwards only that. The base has no model: a backend that serves a nameable model takes model= as its own setting (all three built-ins do) and reports it through model_info.name, which is what tracing hooks label spans with; one that serves whatever its server loaded leaves the name None. The retry wrappers come from the base, which retries a failed call only on its retryable_exceptions (RateLimitError, ServerError and APIConnectionError, timeouts included, from jaz.exceptions by default): raise a transient failure as one of those, or add your own class to that tuple, and anything else fails at once. A user can retry more errors with RetryPolicy(extra_retryable_exceptions=...).
Cost is yours to report. Set cost_usd if you can price the call — LiteLLM's pricing helpers (litellm.completion_cost, litellm.model_cost) are what the built-in LiteLLM backend prices with, and litellm is a JAZ dependency, so it is already installed.
If your backend can price calls, say so with ModelInfo(can_report_cost=True) — the default is False, because JAZ itself prices nothing. Leaving it alone is safe: a cost_budget is then rejected before any call with an error naming your backend, and calls_budget still works. max_input_tokens is the same shape — it defaults to None ("unknown", not "unlimited"), and the ContextWindowWarning hook sizes its window from it, so a backend that does not report one just leaves that hook disabled rather than erroring. Retry is one value too: MyLLM(model=..., retry_policy=RetryPolicy(max_retries=2)).
When a config file names your backend, every key in its block is a constructor argument: model and base_url reach the named parameters, and anything else (temperature) lands in your **query_params tail — a backend declares its settings just by declaring __init__. Whoever loads that file supplies the name → class mapping (the eval harness does it with the non-public jaz._instantiate.build_config(..., resolvers={"llm": my_lookup})); JAZ core holds no registry, and a backend class carries no name of its own. Constructing it yourself needs no name at all: ConfigOverride(llm=my_llm).
Testing without a provider. MockLLM is a backend that answers from a callable, so an agent can be exercised with no API key, no network and no nondeterminism — through the same configure/ConfigOverride as any backend, with retry and cost accounting intact:
from jaz.llm import MockLLM
def scripted(messages):
return "return 42" # a str is the assistant's reply; or return an LLMResponse
with jaz.ConfigOverride(llm=MockLLM(scripted)):
assert jaz.invoke(task="add 40 and 2") == 42
LiteLLM is the default; GitHubCopilotLLM and SGLangLLM ship alongside it for the cases it does not cover, and all three are covered by the stable API. Reach OpenAI and Anthropic through litellm's openai/… / anthropic/… routes — the hand-written vendor-direct backends were removed before the first release. To add or replace a backend, subclass BaseLLM; add it to your loader's map only if a config file needs to name it.
The configured model id is sent verbatim: JAZ neither strips nor rewrites a provider/ prefix, so write the id the backend's own documentation uses. A backend is chosen by configuring it, not by prefixing the model — the prefix tells the configured backend which provider to route to, and a backend that does not route (SGLangLLM) sends the id to its own server verbatim.
Architecture
jaz/
├── _invoke.py # invoke() / ainvoke(), re-exported from `jaz`
├── _config.py # configure() / ConfigOverride / get_config()
├── _agent.py # The REPL loop that drives one invoke
├── repl/ # REPL implementations (PythonREPL)
├── hooks/ # Hook system: events, effects, dispatcher, built-in hooks (BudgetPool, ...)
├── llm/ # LLM backends (LiteLLM by default)
└── protocol/ # Wire-format codec between the LLM and the REPL
Modules with a leading underscore are non-public; everything supported is exported from jaz
or one of its public modules (jaz.hooks, jaz.llm, jaz.repl, jaz.protocol,
jaz.exceptions, jaz.credentials).
Core Concepts
- Agent loop: Orchestrates LLM queries and REPL execution
- REPL: Executes agent-generated code with safety controls
- Hooks: Event-based system for logging, budget control, workflow capture, and extensibility
- Tools: Plain functions passed as inputs; their signature and docstring become the description the agent reads
Built-in Hooks
| Hook | Purpose |
|---|---|
ReturnType |
Declare + enforce the invoke's return type |
ValidateReturn / ValidateREPLCode |
Validate the return value / veto REPL code before it runs |
BudgetPool |
Shared LLM cost / call-count budget with hard-stop enforcement |
IterationLimit / RecursionLimit |
Per-level turn cap / invoke-nesting cap |
BudgetForcing |
Refuse early finishes so the agent keeps working |
ContextWindowWarning |
Warn the agent as its prompt nears the model's window |
PrintLogger / FileLogger |
Log events to console / file |
TrajectoryRecorder |
Write the run as an ATIF trajectory (replay/cost source) |
TrajectoryDirectoryRecorder |
Write the run as a browsable Markdown directory, live (tail -f a running agent) |
TrajectoryReplay |
Resume a run from an ATIF trace: saved responses replay, then live calls take over |
RolloutRecorder |
Record token-native rollouts for training |
JaegerTracing / LangfuseTracing |
OpenTelemetry tracing presets |
Writing Custom Hooks
Hooks observe agent execution via events (from jaz.hooks.events) and influence it by
returning effects (from jaz.hooks.effects):
from jaz import invoke
from jaz.hooks import Hook
from jaz.hooks.effects import AddMessages, Effect
from jaz.hooks.events import LLMQueryEnter
class ConciseHook(Hook):
"""Instruct the agent to be concise after iteration 3."""
# Override the typed per-event handler for the event you care about — no
# isinstance/dispatch boilerplate. (A cross-cutting observer that wants *every*
# event overrides `on_any` instead.)
def on_llm_query_enter(self, event: LLMQueryEnter) -> list[Effect]:
if event.iteration >= 3: # iterations are 0-based
return [AddMessages([{"role": "user", "content": "Be concise."}])]
return []
# Hooks activate via context managers and propagate to nested invoke() calls
with ConciseHook():
result = invoke(task="Solve this step by step")
Hooks compose naturally as context managers:
# Stack multiple hooks
with ConciseHook(), FileLogger("agent.log"):
result = invoke(...)
Event Types
Events fire around three spans — the whole Invoke, each turn's LLMQuery, and each
turn's REPLExec — and every span walks the same four stages. All are importable from
jaz.hooks.events.
| Stage | Observes | Accepts | Fires |
|---|---|---|---|
*Enter |
the proposal | edit effects + Abort |
always |
*Send |
the committed input | supply effects + Abort |
iff the input committed |
*Complete |
the raw result | modify effects + Abort |
iff a raw result exists |
*Exit |
the outcome union (Completed | Aborted | Failed) |
nothing (observation-only) | whenever the span opened |
Plus LLMQueryRetry, fired per retry attempt of an LLM call. Every event carries
contextual data (e.g. LLMQueryEnter.iteration, event.config) and a
timestamp (its emission time — durations are arithmetic over two events' timestamps).
Effect Types
Effects are returned by hooks to influence execution. All are importable from
jaz.hooks.effects; each composes at a specific stage (an effect returned anywhere else
raises InvalidEffectError).
| Effect | Stage | Influence |
|---|---|---|
AddInputs / DropInputs |
InvokeEnter |
Add/un-pass invoke inputs (prompt + REPL) |
DisableRecursion |
InvokeEnter |
Withhold the recursive invoke tool |
SupplyInvokeResult |
InvokeSend |
Supply the invoke's result, skipping the agent loop |
ModifyInvokeResult |
InvokeComplete |
Replace the invoke's result |
AddMessages / DropMessages |
LLMQueryEnter |
Edit the messages sent to the model (transient or persistent) |
SupplyLLMResponse |
LLMQuerySend |
Supply the response, skipping the API call |
ModifyLLMResponse |
LLMQueryComplete |
Replace the model's response |
AddToNamespace / DropFromNamespace |
REPLExecEnter |
Bind/unbind REPL namespace names for the turn |
InsertCode / DeleteCode |
REPLExecEnter |
Edit the code about to run |
SupplyExecResult |
REPLExecSend |
Supply a result, skipping execution |
ModifyExecResult |
REPLExecComplete |
Replace the raw execution result |
Abort |
any Enter/Send/Complete, and LLMQueryExit / REPLExecExit |
Abort the invoke; its error propagates out of invoke() |
BlackboardWrite |
any event | Write to the per-invoke cross-hook blackboard |
Langfuse Tracing Quick Start
from jaz import invoke
from jaz.hooks import LangfuseTracing, ReturnType
with LangfuseTracing():
result = invoke(ReturnType(int), task="Calculate 2+2")
Environment variables:
LANGFUSE_PUBLIC_KEYLANGFUSE_SECRET_KEYLANGFUSE_HOST(optional, default:https://cloud.langfuse.com)
Contributing
This repository is generated: each release publishes a cleaned copy of the package, so changes are not made here. Contributor setup and code conventions live with the source.
License
See LICENSE file.
Metadata
Release files for jaz-lang 0.2.0b1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| jaz_lang-0.2.0b1.tar.gz | 744.7 kB | With provenance | Details for jaz_lang-0.2.0b1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| jaz_lang-0.2.0b1-py3-none-any.whl | Python 3 | none | any | Details for jaz_lang-0.2.0b1-py3-none-any.whl |
Total release size: 1.5 MB
Release files / jaz_lang-0.2.0b1.tar.gz
| Download URL | jaz_lang-0.2.0b1.tar.gz |
|---|---|
| Size | 744.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e3b81093635bea97805e4612568de08a7d1bbce1c77f98208745a94f2d87fcfc
|
|
BLAKE2b-256 checksum How to use checksums |
4706213eef6d189b6c13b27567b45a33cc9b395e5b4c1d14127c3085c5d57d98
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / jaz_lang-0.2.0b1-py3-none-any.whl
| Download URL | jaz_lang-0.2.0b1-py3-none-any.whl |
|---|---|
| Size | 794.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0f50da5fdedd836e4a62939164db3a0ba6892340013fef16a031ee993b751442
|
|
BLAKE2b-256 checksum How to use checksums |
79437f7d77794976d467037b823e1319cd7e8e1d8f83cbbbe62521452813e4ec
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logSecurity for jaz-lang 0.2.0b1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for jaz_lang-0.2.0b1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.