Skip to main content

EBaMl

BAML for Explainable Boosting Machines. Python extracts the graphs from an EBM, and BAML holds the prompts, the LLM clients and the chain-of-thought pipelines.

EBaMl is based on TalkToEBM and adds local explanations (why did the model predict this for this input?) computed exactly and put in words for a layperson or a data scientist, a benchmark that scores answers automatically, a configurable LLM provider, token budgets and typed, structured logging.

LLM calls go through OpenRouter's free router (openrouter/free), so no paid API key is needed.

Supported models

Feature Binary classifier Regressor Multiclass classifier Interactions
Graph descriptions (describe_graph) ✅ ✅ ❌ ⚠️
Model summary (describe_ebm) ✅ ✅ ❌ ⚠️
Graph text (EBMGraph) ✅ ✅ ❌ ⚠️
Local explanations (explain_instance) ✅ ❌ ❌ ❌

✅ supported. ⚠️ partly supported: the main effects are described, and the interaction terms are skipped without a warning. ❌ not supported yet. Full support is on the roadmap.

Setup

Requirements

  • Python 3.13 or later and uv.
  • The BAML toolchain 0.23 or later (step 1).
  • An API key for an LLM provider. The default is OpenRouter's free router, which needs no paid plan. You can use another provider instead, see "Choosing the model".

1. Install the BAML toolchain

curl -fsSL https://pkg.boundaryml.com/install.sh | sh -s

The installer prints a line to add to your shell profile. Add it, then open a new shell:

. "$HOME/.baml/env"

Check it:

baml --version   # toolchain 0.23 or later

If baml resolves to an older install (Homebrew baml, or baml-py in a virtualenv), remove it or put ~/.baml/bin first on PATH. The old CLIs have no describe, generate add or agent commands.

If a command stops with "the BAML agent skill is required", run baml agent install and retry.

2. Get the code and install the Python dependencies

git clone <repository-url> EBaMl
cd EBaMl
uv sync

3. Set the API key of your LLM provider

For the default provider, OpenRouter:

  1. Create an account at https://openrouter.ai and open https://openrouter.ai/settings/keys.
  2. Create a key. Keys start with sk-or-.
  3. Export it in the shell you run the code from:
export OPENROUTER_API_KEY=sk-or-...

The key is read from the environment when a request is sent. It is never written to a .baml file.

Using another provider? Set its variable instead (ANTHROPIC_API_KEY, OPENAI_API_KEY, ...) and pass model= as described in "Choosing the model". Local models (Ollama) need no key.

4. Generate the Python SDK

baml generate

This writes the src/baml_sdk/ package from baml_src/, next to ebaml, so it imports wherever the project is installed. Rerun it every time a .baml file changes, including when you change the provider.

If src/baml_sdk/ does not exist, ebaml.llm.descriptors runs baml generate when you import it. If the package was generated by another BAML toolchain than the one that baml-bridge expects (a version skew), the import runs baml toolchain use <version> and baml generate. If the agent skill check stops baml generate, the import runs it again with --agent-skill-check off. If none of this works, SDKUnavailableError lists the steps that fix the cause.

5. Check the setup

OPENROUTER_API_KEY=unused baml test -x root::Live   # offline tests on a scripted client, no network
baml run main                         # prints "hello from baml"

These two do not call an LLM, so they pass without a working key or provider. They check the toolchain and the SDK.

6. Live test (optional)

Makes real requests to your provider (9 on the OpenRouter free router, so it can be slow or rate limited). It does nothing without OPENROUTER_API_KEY, so with another provider run the example under "Usage" instead.

OPENROUTER_API_KEY=sk-or-... uv run python scripts/live_openrouter.py

It fits a small EBM, runs the three-turn chain directly and checks that the provider reports token usage, then runs describe_graph and describe_ebm. It exits non-zero on the first failed assertion.

Troubleshooting

Symptom Cause and fix
baml: command not found The profile line from step 1 is not loaded. Open a new shell or run . "$HOME/.baml/env".
unknown command: describe (or generate add, agent) An older baml is first on PATH. See step 1.
SDKUnavailableError, or ModuleNotFoundError: baml_sdk The SDK is not generated or has the wrong version, and ebaml could not fix it. The message of the error lists the steps. Run baml generate (step 4).
MissingAPIKeyError The key of the provider is not exported in this shell. The message names the variable. See "Choosing the model" for the variable of each provider.
APIKeyRejectedError The provider rejected the key. Check the key, its account and its credit.
RateLimitedError The provider limited the rate. Lower the concurrency, describe_ebm(ebm, max_concurrency=2), or pin a paid model.
ProviderUnreachableError A request did not reach the provider. Check the network, HTTPS_PROXY and base_url.

Usage

from interpret.glassbox import ExplainableBoostingClassifier

from ebaml.llm.descriptors import describe_ebm, describe_graph
from ebaml.llm.options import PromptOptions

ebm = ExplainableBoostingClassifier().fit(X, y)

print(describe_graph(ebm, feature_index=0))
print(describe_ebm(ebm, num_sentences=30, prompt=PromptOptions(dataset_description="...")))

A graph can be turned into text with EBMGraph.to_text and back with EBMGraph.from_text, which also reads a graph embedded in a longer prompt. It needs the confidence bounds, and it only reads continuous features.

Three keyword-only arguments set the options. graph=GraphOptions(...) sets the options of EBMGraph.from_ebm and EBMGraph.to_text, for example max_tokens. prompt=PromptOptions(...) sets the descriptions that the prompts use (expert_description, graph_description, dataset_description and task_description). model= selects the LLM, see "Choosing the model". The classes are in ebaml.llm.options. Token usage reported by the provider is logged at INFO on the ebaml.llm.descriptors logger.

Explaining one prediction

from ebaml.llm.descriptors import describe_ebm, explain_instance

ebm = ExplainableBoostingClassifier(interactions=0).fit(X, y)  # binary classifier, no interactions
global_explanation = describe_ebm(ebm)  # computed once, prepended to every local prompt

explanation = explain_instance(ebm, X.iloc[0].to_dict(), global_explanation, audience="layperson")
print(explanation.counterfactual)      # what would have to change to flip the prediction
print(explanation.sufficient_reason)   # which inputs settle the prediction on their own
print(explanation.completions)         # how certain the outcome is as inputs become known

The facts are computed exactly from the scores and confidence intervals of the EBM (ebaml.local); the LLM only puts them in words. They are the three queries of arXiv:2010.12265: minimum change required, minimum sufficient reason and count completions. audience="layperson" (default) gives no model numbers; audience="data_scientist" quotes scores, probabilities and confidence intervals. It supports binary classifiers without interactions, and explanation.facts holds the numbers behind the text. The "Local explanations" page of the documentation explains what each explanation means, the options and the limits, and notebooks/local_explanations.py walks through them on the Titanic data.

Errors

Every error that ebaml raises on purpose derives from ebaml.EBaMlError. Use except EBaMlError to catch the errors of ebaml apart from the errors of other code. Each error for a bad value also derives from ValueError, so an except ValueError catches it too.

Error ValueError Raised when
InvalidOptionError yes An option is not one of its valid values, for example an unknown Normalization.
OptionRangeError yes A number option is outside its range, for example confidence_level=1.
InvalidInstanceError yes The instance to explain lacks a value, or has a category that the EBM did not see.
UnsupportedModelError yes A local explanation gets an EBM with interactions, or one that is not a binary classifier.
GraphTextError yes EBMGraph.from_text cannot read the text of a graph.
EBMGraphError no The base class of the errors of a graph.
EBMGraphShapeError yes The bins of a graph do not fit its scores, or a boolean format gets more than two bins.
EBMGraphTokenLimitError no The text of a graph is over the token budget, and its format cannot be simplified.
EBMGraphSimplificationError no The text of a graph is still over the token budget at the largest simplification.
SDKUnavailableError no baml_sdk is missing, and ebaml cannot run baml generate. The error also derives from ImportError.
LLMRequestError no The base class of the errors of a provider. Its message lists the steps that fix the cause.
MissingAPIKeyError no A provider needs an API key and the environment has none.
APIKeyRejectedError no A provider answers a request with "unauthorized" or "forbidden".
RateLimitedError no A provider limits the rate of the requests.
ProviderUnreachableError no A request does not reach the provider.

EBMGraphShapeError, EBMGraphTokenLimitError and EBMGraphSimplificationError derive from EBMGraphError. The four provider errors derive from LLMRequestError. All classes are in ebaml.core.exceptions, and ebaml exports them too.

How it works

Where What
src/ebaml/core/ Shared kernel: types and errors. It imports no other part of ebaml.
src/ebaml/graphs/ Graph extraction and simplification (numpy, interpret), and the text the LLM reads.
src/ebaml/llm/ The edge: the calls into BAML (descriptors.py), their options and the retry after a cut-off reply.
baml_src/talk_to_ebm.baml Prompts, the default LLM clients, the three-turn description chain, the summary, describe_ebm, token accounting.
baml_src/model_spec.baml The model of a call: ModelSpec and one client builder for each provider.
src/ebaml/local/, baml_src/local_explanations.baml Local explanations: the facts of one prediction computed deterministically, their rendering as JSON texts, and the prompts that put them in words.
baml_src/talk_to_ebm_test.baml, baml_src/model_spec_test.baml Offline tests on a scripted client.
baml.toml Generator config for the Python SDK.
src/ebaml/benchmark.py, benchmark/ The benchmark: loading, answer parsing and scoring, and the data and runner (see Benchmark).
notebooks/ Example notebooks, written with marimo.
docs/, zensical.toml The documentation, built with Zensical (see Development).
tests/unit/ Python unit tests (100% coverage required). scripts/live_openrouter.py is the optional live test.
scripts/ Maintenance scripts, see scripts/README.md.
.github/ CI workflow and the scripts that turn failures into reports.
src/baml_sdk/ Generated. Do not edit.
  • Model: openrouter/free picks a free model for each request, so the model can change between calls and between the turns of one conversation. Quality and speed vary with it.
  • Retries: transient failures (rate limits, network errors, 5xx) are retried up to 3 times. Requests the provider rejects are not retried.
  • Concurrency: describe_ebm describes features concurrently, at most max_concurrency at a time (default 8). Free models are rate limited, so lower it (describe_ebm(ebm, max_concurrency=2)) if you see rate-limit errors.
  • Token budget: there is no tokenizer before a call, so the check uses a rough estimate (about 3 characters per token). Going a little over max_tokens only warns. Going more than overbudget_percent (default 50) over makes to_text simplify the graph, or raise if it cannot. The provider's own error is the real limit. The functions in ebaml.llm.descriptors pass the BAML check. A direct call of EBMGraph.to_text checks nothing, unless you pass check_budget, for example baml_sdk.check_token_budget.

Notebook

notebooks/titanic.py is a marimo notebook that ports TalkToEBM's example. It trains an EBM on the Titanic dataset (downloaded from OpenML), asks the LLM to describe a graph and the whole model, and shows the graph and text API (from_ebm, simplify, to_text, from_text). The LLM cells run only when you press their button. They use the default model, which needs OPENROUTER_API_KEY. See "Choosing the model" for other models.

uv run marimo edit notebooks/titanic.py   # interactive
uv run marimo run notebooks/titanic.py    # as an app

notebooks/local_explanations.py is the place to learn the local explanations: pick a passenger, see the facts with no API key, move the confidence slider, check the guarantees against random completions, compare what the LLM reads for a layperson and a data scientist, and (with a key) get the explanations in words.

uv run marimo edit notebooks/local_explanations.py

Benchmark

The benchmark follows TalkToEBM's benchmark that checks how well a model reads EBM graphs: function recognition (easy and hard), the largest jump, monotonicity, reading a value and the widest confidence interval. It makes real requests, and it asks two LLM turns per question (6 tasks, 480 questions in all). The default model needs OPENROUTER_API_KEY. The flag --model provider/model selects another model.

uv run python benchmark/run.py --task monotonicity --limit 10 --output results.json

See benchmark/README.md for the tasks, the scoring and what it leaves out.

Choosing the model

The default model is OpenRouter's free router, openrouter/free. It needs OPENROUTER_API_KEY. The model argument of describe_graph, describe_ebm and explain_instance selects another model for one call. EBaMl sends every LLM request of that call to that model. You do not change the package.

Quick start

Pass the shorthand "provider/model":

from ebaml.llm.descriptors import describe_ebm

print(describe_ebm(ebm, model="anthropic/claude-haiku-4-5"))

Pass ModelOptions to set more:

from ebaml.llm.options import ModelOptions

model = ModelOptions(provider="anthropic", model="claude-haiku-4-5", temperature=0.2, max_output_tokens=8000)
print(describe_ebm(ebm, model=model))

EBaMl splits the shorthand at the first /. The text "openrouter/openai/gpt-4o-mini" selects the provider openrouter and the model openai/gpt-4o-mini. EBaMl checks the provider before the first paid request. It raises InvalidOptionError if the provider is not in the table below.

Providers

The provider is a BAML provider prefix. Set the environment variable of the key in the shell that runs Python.

Provider Default key variable Notes
openrouter OPENROUTER_API_KEY The model name is the OpenRouter name, for example openai/gpt-4o-mini.
openai OPENAI_API_KEY
anthropic ANTHROPIC_API_KEY
google GOOGLE_API_KEY The Gemini API.
ollama None A local server. base_url defaults to http://localhost:11434/v1.
openai-generic None Any OpenAI-compatible server. base_url is required. Set api_key_env if the server needs a key.
azure AZURE_OPENAI_API_KEY The model name is the name of the deployment. Set base_url to the deployment, or set AZURE_OPENAI_ENDPOINT.
vertex None Google Cloud credentials (GOOGLE_APPLICATION_CREDENTIALS or gcloud auth). The project comes from GOOGLE_CLOUD_PROJECT.
bedrock None The AWS credentials of the machine. The region comes from AWS_REGION. api_key_env has no effect.

Run baml describe <client>.new, for example baml describe aws.BedrockClient.new, to see all settings of a client.

Options

Field Meaning Default
provider The BAML provider prefix from the table. Required.
model The name of the model at the provider. Required.
api_key_env The name of the environment variable that holds the key. The variable of the provider.
base_url The address of the server. The address of the provider.
temperature The sampling temperature. 0.7 (0.0 in the benchmark).
max_output_tokens The most tokens of one reply. See below.
headers Headers to add to each request. None.

A field that you set wins over the default of the call. The output limit has two defaults:

Calls max_output_tokens
Turn 1 of a graph description, the draft of the summary, and the two turns of the benchmark 8000
All other calls, including the local explanations 6000

Examples

# OpenRouter, a pinned model (the free router picks a new model for each request)
describe_ebm(ebm, model="openrouter/openai/gpt-4o-mini")

# OpenAI
describe_ebm(ebm, model="openai/gpt-4o-mini")

# Google Gemini
describe_ebm(ebm, model="google/gemini-2.5-flash")

# A local model with Ollama. It needs no key.
describe_ebm(ebm, model="ollama/llama3.1")

# A company gateway with an OpenAI-compatible API
gateway = ModelOptions(
    provider="openai-generic",
    model="my-model",
    base_url="https://llm.example.com/v1",
    api_key_env="MY_GATEWAY_KEY",
    headers={"x-team": "ebm"},
)
describe_ebm(ebm, model=gateway)

The benchmark has the flag --model:

uv run python benchmark/run.py --model anthropic/claude-haiku-4-5 --limit 10

Keys

Put the key in an environment variable, in the shell or with os.environ. ModelOptions holds the name of the variable, never the key, so the key does not appear in a repr, a log line or a traceback. EBaMl reads the variable only when it sends a request.

Limits

  • One call uses one model. The turns of a call cannot use different models.
  • Python cannot set up fallback chains or round robin. A BAML client object cannot cross from Python to BAML, so Python sends plain data and BAML builds the client. A BAML client declaration can use ai.clients.Fallback: see "Change the default model" in CONTRIBUTING.md.
  • A reasoning model counts its hidden tokens in max_output_tokens. Keep the limit large, or the reply can end before the answer.
  • Quality, speed and cost depend on the model. Small models can ignore the number of sentences that you ask for.

Every client has the retry policy of the default clients: EBaMl retries a transient failure up to 3 times.

Development

baml fmt          # format .baml files
baml check        # compile check
baml test -x root::Live   # run the offline BAML tests
baml generate     # regenerate src/baml_sdk/
uv run pytest     # Python unit tests, fails under 100% coverage
uv run pytest README.md docs src --no-cov   # the code examples of the docs and the docstrings
prek run --all-files                       # every hook: formatters, ruff, vulture, pyrefly, baml fmt, ...
uv run python scripts/mutation.py          # mutation testing of all modules, fails if any mutant survives
uv run python scripts/mutation.py --base origin/main   # the same, but only for the lines changed since main
uv run python scripts/mutation.py --module src/ebaml/local/facts.py   # the same, but only for one module
uv run python -m scripts.check_ignores     # every lint/type/test workaround needs a reason written next to it
uv run python -m scripts.compile_ignores   # all workarounds and their reasons as one Markdown report to audit
uv run --group docs python -m scripts.build_docs         # build the docs; fails on a broken link or a docstring warning
uv run --group docs python -m scripts.build_docs serve   # preview the docs (exports the notebooks, so the first run is slow)

CI (.github/workflows/ci.yml) runs prek, secret and security scans (trufflehog, semgrep, trivy), baml check and baml test, pytest with 100% coverage, the code examples of the documentation, and, in pull requests, mutation testing of the changed lines with a 100% score. A weekly run tests all modules, one job for each module. Lint and type workarounds (# noqa, # type: ignore, skipped tests, ...) must carry a written reason next to them.

The python code blocks of README.md and docs/*.md run in order, and the blocks of one file share their variables. The Examples sections of the docstrings run as doctests. tests/docs/conftest.py gives each Markdown file a small data set as X and y, and it replaces the LLM calls with fakes, so the examples send no request. Put setup that the reader does not need in an <!-- invisible-code-block: python --> comment. To skip a block, put <!-- skip: next --> above it, and write the reason in the text.

Metadata

Release files for ebaml 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ebaml 0.1.1
File Size Uploaded Provenance Actions
ebaml-0.1.1.tar.gz 1.6 MB Without provenance Details for ebaml-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ebaml 0.1.1
File Interpreter ABI Platform Actions
ebaml-0.1.1-py3-none-any.whl Python 3 none any Details for ebaml-0.1.1-py3-none-any.whl

Total release size: 3.2 MB

Release files / ebaml-0.1.1.tar.gz

Download URL ebaml-0.1.1.tar.gz
Size 1.6 MB
Tags Source
SHA-256 checksum
How to use checksums
1967e4a3c9b0e0e82c0d2ad6fdd34854910b8e7291f03a1457bf046cfdb634d4
BLAKE2b-256 checksum
How to use checksums
b5789cc12f619bd86c1a9240642eb54d88b8292570f26ae1eb54e5fdfcea61da
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / ebaml-0.1.1-py3-none-any.whl

Download URL ebaml-0.1.1-py3-none-any.whl
Size 1.6 MB
Tags Python 3
SHA-256 checksum
How to use checksums
233c1eee9400a7667eb08aac180e7e772c18c8f9314b2d0c54b7b7dd9a0ed4f1
BLAKE2b-256 checksum
How to use checksums
825a67400a4a6d1e37605c2fb42aff50325356fee94dfd4ff06336a03d88eb86
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for ebaml 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page