zoo-framework 0.14.0

Python declarative multi-task orchestration framework powering next-gen Agents and workflows

English | 中文

Zoo Framework runs long-lived background tasks in-process — no broker (no Redis / RabbitMQ), no distributed setup, designed so that AI-agent-generated code can drive it: a narrow extension surface, loud failures, and a spec baseline an agent can verify by running tests.

It is for developers who need scheduled, observable, persistent background work embedded in their own process — the in-process alternative to Celery, not a replacement for it.

The problem it solves

Wiring background work out of raw threading looks like twenty lines and turns into a few hundred: who owns the thread, what stops the same job from overlapping itself, how does a stuck job get noticed, what happens to in-flight work on shutdown, where does the state go. Each of those is a place to be subtly wrong, and the failure mode is usually silence — a job that stopped running, or two that ran at once.

Zoo Framework takes those decisions out of your code:

• Overlap — a Worker that is still in flight is skipped for the round, never dispatched twice concurrently.
• Stuck work — run_timeout observes and circuit-breaks. Note the framework does not force-kill a running Worker; it stops dispatching it.
• Shutdown — Master.shutdown() stops dispatch, cancels the schedule, stops the event pipeline and monitoring, then unregisters every Worker — which is where the state machine's final save happens.
• State — periodic and on-shutdown persistence, atomic replace, rolling backups.
• Errors — unsupported inputs raise rather than silently degrade.

The whole thing in 30 seconds

Save as main.py and run with Python 3.11+:

from zoo_framework.core import Master
from zoo_framework.workers import BaseWorker


class MyWorker(BaseWorker):
    """A task unit that runs in a loop."""

    def __init__(self):
        super().__init__(
            {
                "is_loop": True,  # keep running across scheduling rounds
                "delay_time": 1.0,  # seconds to wait after each execution
                "name": "MyWorker",
                # "run_timeout": 30,  # optional: observe + circuit-break after 30s
            }
        )
        self.counter = 0

    def _execute(self):
        self.counter += 1
        print(f"Hello from MyWorker! Count: {self.counter}")


if __name__ == "__main__":
    master = Master()
    # Registered workers join the schedule. Master() alone only runs the two
    # built-in system workers (EventWorker / StateMachineWorker).
    master.register_worker("MyWorker", MyWorker)
    master.run()

Expected output: a Hello from MyWorker! Count: N line every second. (When piping stdout to a file, CPython's default block buffering delays the lines — run in a terminal, or use python -u, to see them as they happen.)

You declare task units ("Workers"), register them, and the framework keeps them running: it decides when each one is dispatched, keeps concurrent instances apart, observes how long they take, and shuts down cleanly with state on disk.

It is not a web framework and not a task queue — there is no HTTP layer, no broker, and no distributed scheduling. It is the in-process equivalent: a scheduler, an event pipeline, and a state store that you embed in a service.

How it compares

Zoo Framework is one point in a space that already has good tools. It is the right choice when the work is long-lived, in-process, and not worth a broker:

Read the table as a scope statement, not a scoreboard: Celery is the right answer the moment work must cross machines, and APScheduler is the right answer for cron. Zoo Framework deliberately declines both, and the row it wins on is "no external dependency, no broker, no cron daemon — but still scheduled, still observable, still persistent." The comparison reflects each project's general positioning; no cross-project benchmark was run.

Built for AI-agent-generated code

The extension surface is deliberately narrow, so generated code is short, verifiable, and — when it is wrong — wrong loudly.

Three steps, no glue code

class OrderSyncWorker(BaseWorker):  # 1. subclass
    def __init__(self):
        super().__init__({"is_loop": True, "delay_time": 5, "name": "OrderSync"})

    def _execute(self):  # 2. write only the business logic
        sync_orders()


master.register_worker("OrderSync", OrderSyncWorker)  # 3. register

Thread management, concurrency limits, in-flight de-duplication, timeout circuit-breaking, graceful shutdown and state persistence are the framework's job. An agent does not have to generate that code — and therefore cannot get it wrong.

Configuration is separate from implementation

A Worker only depends on the props dict handed to __init__. It has no visibility into framework internals, so generating one does not require reading the source or understanding how Waiter / WorkerRegistry / EventReactor relate.

Failures are explicit, never silently swallowed

This matters more for generated code than for hand-written code: an agent cannot detect a silent downgrade, and a loud error is the signal it needs to self-correct.

Changes can be checked automatically

The framework carries a spec baseline and a continuously-passing regression suite covering the core contracts, so an agent's change can be judged by running pytest rather than by asking a human to read it. The Tests badge above reflects the current suite status.

There is a working consumer, not only a claim

example/agent is a minimal coding-agent loop built on the released package (currently >=0.12,<0.13): a prompt goes in, tool calls are dispatched through the event pipeline, and session state is persisted by the framework's state machine — so a session resumes after a process restart, and a replayed tool call does not fire its side effect twice. Write and exec calls pass a fail-closed permission gate on every call, and code evaluation is never exposed. It is the consumer that turned this section's statements into measurements: the gaps it found are issues #72 / #73 / #74, and #72 / #73 are fixed in 0.12.0.

Installation

pip install zoo-framework

Requires Python 3.11+ (the floor is backed by evidence, in the 「Python 下界的依据」section of the development guide).

Quick Start

The minimal example above is the quick start — the same code checked in as example/minimal.py (copy → python example/minimal.py). To scaffold a project with a config, a directory layout and a demo Worker already registered, use the CLI instead:

zfc --create myapp
cd myapp
python src/main.py

The scaffolded project has a demo Worker wired in — a [sample_worker] tick #N line every 10 seconds. That is your code running; the framework's own system logs look different (they go through the logger). Adding your own worker is optional — the block is self-contained (its first two lines are the ones you just ran), so you can copy it as-is:

zfc --create myapp
cd myapp
zfc --worker my_task

There is no separate config file to write — Master() defaults to ./config.json relative to the working directory, and the minimal example above runs without one.

A Worker must be registered as a class. WorkerRegistry validates with issubclass, so a function or an instance is rejected with TypeError: issubclass() arg 1 must be a class. This holds independently of @cage: any wrapper that replaces the class with a factory function fails the same way, and the rule still stands now that @cage is removed. Process-level sharing is declared through the container instead.

How it runs

Master.run()
   └─ scheduling round (every 1s)
        ├─ for each registered Worker:
        │    ├─ timed out?        → circuit-break, stop dispatching it
        │    ├─ still in flight?  → skip this round
        │    └─ otherwise         → dispatch to a thread / the pool
        └─ on completion: unregister in-flight, report result to the event pipeline

Workers execute concurrently within a round and are never dispatched twice at the same time. Master.shutdown() stops dispatch first, then cancels the scheduling task, stops the event loop, stops monitoring, and finally unregisters every Worker — which is where the state machine's last save happens.

How the pieces connect:

flowchart TB
    cfg["config.json"] --> pf["ParamsFactory<br/>resolved once, at first import"]
    m["Master"] --> pf
    m --> wr["WorkerRegistry<br/>module-level singleton"]
    m -->|"worker:mode"| w["Waiter<br/>ThreadPerTaskModel / ThreadPoolModel"]
    w -->|"dispatch, once per round"| wk["Worker._execute()"]
    wk --> set["WorkerDispatchCore.settle()<br/>single settlement point:<br/>unregister in-flight, report result"]
    set --> pipe["Event pipeline<br/>EventChannel → EventFIFO → Reactor"]
    m --> sm["StateMachineWorker<br/>periodic save, atomic replace + rolling backup"]

Features

Core concepts

The framework is named after a zoo, but the metaphor only affects naming, not semantics — if a name is unclear, read the right-hand column:

Event pipeline

from zoo_framework.core.aop import event


@event(topic="order.created", channel="business")
def on_order_created(req):
    print(req.topic, req.content)

Events are isolated per channel; an event can select its response mechanism (first responder only / by priority / all / a named responder). Reactor registration is idempotent — registering the same object twice does not append it twice.

State machine

Read and write by "scope + key path"; key paths support dotted nesting:

from zoo_framework.statemachine import StateMachineManager

sm = StateMachineManager()

sm.set_state("order", "status", "pending")  # first write creates the scope
sm.set_state("order", "status", "paid")  # overwrite
sm.get_state("order", "status")  # -> 'paid'

sm.set_state("order", "item.count", 2)  # nested key
sm.get_state("order", "item.count")  # -> 2

# Observe changes
sm.observe_state("order", "status", lambda payload: print(payload["value"]))
sm.unobserve_state("order", "status", observer)

State is persisted periodically by StateMachineWorker and once more on shutdown: written to a .tmp file then atomically replaced, keeping the last 5 backups in a sibling backups/ directory.

CLI

# Scaffold a project (produces <name>/src/{main.py,workers,conf,params,events},
# including a demo Worker that runs out of the box)
zfc --create myapp

# Add another Worker of your own (writes src/workers/, and wires it into src/main.py)
cd myapp
zfc --worker my_task

# Run
python src/main.py

There are exactly two options, --create and --worker, and they may be combined in one call (zfc --create myapp --worker my_task puts the Worker into the project being created). Both validate their input before producing anything, and neither accepts a silently ineffective call:

• --create fails and exits if the target directory already exists — it does not merge and does not overwrite. Check for the directory yourself if you need idempotency in a script.
• --worker names must be valid Python identifiers: no leading digit, no hyphens, dots or spaces, no keywords. my-task and 123task are rejected; use underscores (my_task).
• --worker searches the current directory and its parents for a scaffolded project containing src/main.py. If none is found, it fails before writing and tells you to run zfc --create <name> or switch to a scaffolded project.

On failure the command exits non-zero with a reason, leaving no half-written output.

Performance

Measured on Windows, Python 3.13, with a representative workload (JSON encode/decode + string processing). bench/ has the scripts and raw data.

Framework overhead is roughly 100–220 µs per task and is dominated by thread dispatch and the wake-up back to the scheduler. It is therefore negligible for millisecond-scale work and dominant for sub-100 µs work — pick your task granularity accordingly. bench/DECISION.md has the breakdown and the cross-platform caveats.

End-to-end duration of each baseline relative to Zoo Framework (above 1.0 = Zoo faster): at concurrency 1 and the smallest body tier Zoo is 2.27× faster than process_pool, and at parity with apscheduler / asyncio / bare threads once the body is ≳40 µs. Snapshot of the 0.10.0 run — the maintained, version-by-version report lives at zoo-bench.

On the numbers above. They are one-off evidence from the adopt-rust-core study, measured on Windows only — do not compare them against Linux runs. The maintained, version-by-version benchmark lives in the separate zoo-bench repo: live report. It runs on native Linux CI, compares against hand-written baselines and the standard-library concurrency models, publishes its raw data, and reports the tiers where the framework loses.

Documentation

User documentation → https://yearsalso.github.io/zoo-framework-doc/ (bilingual: 中文 / English) — installation, tutorials, core concepts, API reference.

Writing a native task (the optional Rust extension) → docs/native-tasks.md: contract, error families, evolution rules, the contract validator, and a "should this go native?" table.

The docs/ directory in this repository keeps maintainer and contributor documentation (architecture, repository layout, debugging, contribution rules, brand guidelines, performance benchmarks). It is not aimed at people using the framework:

• Architecture — module layout and data flow
• API Reference — generated from docstrings
• Development Guide
• Debugging Guide
• Roadmap

Community & Feedback

• Questions, ideas, bug reports — GitHub Issues is the single channel; there is no chat server or mailing list.
• Security vulnerabilities — do not open a public issue. See SECURITY.md for the private reporting route.
• Code of Conduct — CODE_OF_CONDUCT.md.

The project is maintained by one person. MAINTAINERS.md turns that into numbers you can hold it to: a small pull request or a clear report gets a first reply within 7 days. A report with a reproduction, your Python version and your OS gets a response far faster than one without. Everyone whose work lands is listed in CONTRIBUTORS.md.

Governance

Who decides what, who owns what, and how long a reply is expected to take:

• GOVERNANCE.md — how decisions are made, and how to become a maintainer
• MAINTAINERS.md — areas of responsibility and the expected response times
• ADOPTERS.md — who uses the framework, each row labelled verifiable or not
• SECURITY.md — vulnerability reporting and the trust model
• CODE_OF_CONDUCT.md — expected behaviour and how to report abuse
• CONTRIBUTING.md — the one-page entry point for contributors
• .well-known/security.txt — the machine-readable security contacts (RFC 9116)

This is a one-person project, and GOVERNANCE.md says so plainly instead of describing a committee that does not exist.

Contributing

Contributions are welcome. CONTRIBUTING.md is the one-page entry point; the complete spec is docs/CONTRIBUTING_MAINTAINER.md, and the branch model is in docs/BRANCHING.md. For a scoped first task see docs/GOOD_FIRST_ISSUES.md; contributors are listed in CONTRIBUTORS.md.

git clone --recursive https://github.com/YearsAlso/zoo-framework.git
# (omit --recursive only if you don't need example/agent — see example/README.md)
cd zoo-framework
pip install -e ".[dev]"       # or: uv sync --extra dev (uv.lock is kept in sync)
pre-commit install
pytest                        # see the Tests badge for current suite status

pip via pyproject.toml and uv sync via the committed uv.lock are both supported — they resolve the same dependency set. ruff check, ruff format, pytest, mypy and bandit are all hard CI gates.

FAQ

The questions this framework is asked most often. Each short answer links to the long-form answer in docs/FAQ.md.

Can I run scheduled background tasks without Redis or RabbitMQ?

Yes — that is this framework's core scenario. pip install zoo-framework, subclass BaseWorker, register it, and scheduling happens inside your own process: no broker, no cron daemon. Full answer: docs/FAQ.md.

How is it different from Celery, and which should I pick?

Celery runs separate worker processes coordinated by a broker, so tasks can cross machines — that is exactly what Zoo Framework declines. Pick Celery the moment work must leave the process; pick this when the work is long-lived, in-process and not worth a broker. Full answer: docs/FAQ.md.

How is it different from APScheduler?

Both embed in your process with no broker. APScheduler wins on cron expressions and job stores; Zoo Framework adds an event pipeline (priority, retries, dead-letter), in-flight de-duplication, timeout circuit-breaking and atomic state persistence. Use APScheduler if you need cron. Full answer: docs/FAQ.md.

Does it support multiple processes or work across machines?

No. Multi-process execution is not implemented — the mode constants are placeholders and requesting one is rejected — and there is no cross-machine scheduling. If a task must leave the process, use Celery. Full answer: docs/FAQ.md.

Can I use cron expressions?

No. Only a fixed delay_time polling interval exists. If you need cron expressions, use APScheduler. Full answer: docs/FAQ.md.

What happens if a task hangs — is it killed?

It is not killed. A run_timeout observes and circuit-breaks: the timeout is recorded, the Worker is marked unhealthy and stops being dispatched, while the already-running execution keeps going — CPython cannot safely interrupt a running thread. Full answer: docs/FAQ.md.

Is state restored after a restart?

Yes, if you use the state machine (StateMachineManager). State is saved periodically and on shutdown — written to a temporary file and os.replaced atomically, with up to 5 rolling backups — and loaded again on the next start. Full answer: docs/FAQ.md.

Why is it called Zoo, and what do the names mean?

The framework is named after a zoo: Worker (task execution unit), Master (lifecycle entry point), Waiter (the scheduler), Cage (the scoped registry of shared instances). The metaphor only affects naming, not semantics — the core concepts table maps every name to what it does. Full answer: docs/FAQ.md.

What does it have to do with AI agents?

The extension surface is deliberately narrow — subclass, implement _execute(), register the class — failures are loud instead of silent, and the suite lets a change be judged by running pytest. Generated code therefore stays short and can self-correct. A working consumer — a minimal coding agent with restart-resume — lives in example/agent. Full answer: docs/FAQ.md.

Is anyone using it in production?

No verifiable third-party adopter yet. The maintainer's own projects use it, but that cannot be checked from the outside, and ADOPTERS.md labels every row accordingly rather than implying adoption it cannot show. Full answer: docs/FAQ.md.

Does it retry a task that fails?

No — a failed execution is reported on the error path and that round ends. The same Worker is still dispatched on its next scheduled round, but the work that failed is not retried. The only automatic retries here are for events (a reactor carries a retry strategy and a node carries a retry count), not for Worker executions. Full answer: docs/FAQ.md.

License

Apache License 2.0 © XiangMeng