Cairns
A microframework for compute graphs with caching, tracing, and replay.
Think PyTorch for agent pipelines — though nothing about it is agent-specific.
You write ordinary async Python; a @step decorator turns each function into a
tracked, and optionally cached node in a graph that emerges from execution instead of being
declared up front.
Alpha. Public API names, the on-disk cache format, and the higher-order wrappers may still change between minor versions. Pin a version if you depend on it.
Why
Declarative graph frameworks (LangGraph, CrewAI, Airflow-style DAGs) force you into their node/edge DSL. Cairn goes the other way: the graph is your code, the framework just instruments it. From that you get:
- Caching keyed on
(function identity, body version, resolved args)— change one function, only its downstream re-executes. - Tracing — live, structured event log; a built-in TUI renders it.
- Resume — a failed pipeline reruns from the last successful step.
- Replay — cached runs can replay with original timing, indistinguishable from a live execution.
- Human-in-the-loop as a regular async step (
await_input(...)).
Works for LLM pipelines, scrapers, ETL, long-running research — anything that fits into mostly-pure async functions.
Install
pip install cairns[tui] # TUI is worth having
pip install cairns[full] # TUI + pydantic hashing
Or with uv:
uv add cairns --extra tui
Requires Python 3.12+.
Hello world
import asyncio
from cairns import step, run, trace
@step(memo=True)
async def fetch(url: str) -> str:
trace("fetching", state="running")
await asyncio.sleep(0.2) # pretend HTTP
return f"<html>{url}</html>"
@step
async def extract(html: str) -> int:
return len(html)
@step
async def pipeline(urls: list[str]) -> list[int]:
pages = [fetch(u) for u in urls] # returns Handles; runs concurrently
return [await extract(p) for p in pages] # pages are awaited inside extract
run(pipeline(["https://a", "https://b", "https://c"]), store_path=".cairns")
Run it twice. The second run is instant — every @step(memo=True) result is
looked up by cache key. Edit the body of extract, rerun: only extract
re-executes, fetches are cache hits.
The derived body hash is a convenience for quick iteration. For expensive or long-lived memoized steps (LLM calls, slow fetches, anything a scheduled job reuses across weeks), prefer an explicit version and bump it when the step's behaviour or prompt changes:
@step(memo=True, version="2")
async def summarize(text: str) -> str: ...
A declared version makes the memo match on it alone, so reformatting or refactoring unrelated code never throws away results you paid for.
CLI
cairns examples/research_fake_llm.py # run the pipeline, opens TUI if installed
cairns examples/research_fake_llm.py slow # run the `slow` entry point
cairns examples/research_fake_llm.py -f # clear this entry's cache, then run
cairns # interactive run browser over past runs
cairns list # flat list of runs
cairns show [RUN_ID] # print a trace (latest if omitted)
cairns gc [--before YYYY-MM-DD] # garbage-collect old runs
Default store is ./.cairns/. Override with --store PATH (or -s). Entry
points default to a function named main; pass a second positional arg to
pick another, e.g. cairns script.py my_pipeline.
Examples
Each example runs standalone with python examples/<name>.py, or through the
CLI with cairns examples/<name>.py for the TUI.
| Example | What it shows |
|---|---|
scraper.py |
Fan-out + chain, non-AI, fully mocked. Good first look. |
failure_resume.py |
A step fails on item 3; rerun resumes from cache. |
research_fake_llm.py |
Fan-out across N subjects, retry loop, rate limiting, simulated 20% API failure rate. No API key needed. |
hitl.py |
await_input inside a step — TUI input widget, stdin fallback. |
research_haiku.py + claude.py |
Live research over real AI companies via the claude CLI (Haiku). Cached by ISO week — re-runs within the week are free. |
What you'll see
With cairns[tui] installed, the CLI opens a live span tree: each @step
invocation is a row, child steps indent, trace(...) calls attach as
annotations, and cost={...} kwargs get summed up the tree. Failures colour
red, running steps pulse, completed steps show wall time + own time (excluding
waits on children).
Without the TUI, the same events stream to .cairns/runs/{entry}-{ts}/trace.jsonl
and you can read them with cairns show.
Docs
docs/motivation.md— the problem and the analogy to PyTorch.docs/design.md— all primitives, event log, stores, plugin points.docs/patterns.md— comparison against Prefect, LangGraph, Temporal, Flyte, CrewAI across seven patterns.
Status
Alpha, but the core works:
@step,Handle,trace,cached_output/tracing,replayable,rate_limited,await_inputall shipped.- File-backed content-addressed store, JSONL trace sink, symlinked run layout, GC.
- Live TUI for span-tree viewing.
- 110 tests, covering core + hashing + disk + resume + GC + metrics + patterns.
Possible future work: a web UI, distributed execution, distributed cache store.
Feedback and breakage reports welcome via issues.
1 maintainer
Release files for cairns 0.2.11
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| cairns-0.2.11.tar.gz | 284.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| cairns-0.2.11-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 361.0 kB
Release files / cairns-0.2.11.tar.gz
| Download URL | cairns-0.2.11.tar.gz |
|---|---|
| Size | 284.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6934a69836bb8a2c8d9d8fef0e25803dbf8bbbaabf48424af220a32d652abc0b
|
|
BLAKE2b-256 checksum How to use checksums |
1d29677e3ca6e80f5a629182e0c24b7b8c3b950f62f432212aef7dd707fe69b4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / cairns-0.2.11-py3-none-any.whl
| Download URL | cairns-0.2.11-py3-none-any.whl |
|---|---|
| Size | 76.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ed6cceb6cf19ad9dc3a3c753250bc82b956f3bde2add17d6159761df4e72d49e
|
|
BLAKE2b-256 checksum How to use checksums |
6fbb908390651133503e6b975ef3ccd42e77eee0dd7a52e3049addcde4641a63
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
