zef 0.1.54

A Rust-powered Python platform for building reactive, data-driven applications with managed effects, actor-based concurrency, content-addressable functions, and a refinement type system.

Zef 🌿

A Rust-powered Python platform for building reactive, data-driven applications with managed effects, actor-based concurrency, content-addressable functions, and a refinement type system.

Zef treats everything — types, functions, effects, errors — as plain data. Values are contiguous in memory (like FlatBuffers), require no serialization, and cross language and process boundaries without conversion overhead.

What does Zef look like?

Composable Pipelines

from zef import *

names = ['Alice', 'Bob', 'Charlie', 'Dave']

names | filter(Z | length | greater_than(3)) | sort | collect
# ['Alice', 'Charlie', 'Dave']

range(10) | filter(is_prime) | map(multiply(2)) | collect
# [4, 6, 10, 14]

Effects as Data

Side effects are values you construct and then execute. There's a clear syntactic boundary between describing what to do and actually doing it.

# Describe the effect — this is just data
effect = FX.StartHTTPServer(
    routes={
        '/':       ET.HTML(content='<h1>Hello!</h1>'),
        '/api':    ET.JSON(content={'status': 'ok'}),
        '/health': ET.JSON(content={'up': True}),
    },
    port=8080,
)

# Execute it
effect | run

Actors and Streaming

Actors subscribe to topics, receive messages, and produce effects. State is threaded through successive invocations.

from zef import *

topic = ET.Topic('🍃-a0b1c2d3e4f567890abc')   # use the pub-sub system

def handle_msg(msg, state):
    effects = [FX.Print(content=f"[{state}] got: {msg}")]
    new_state = state + 1
    return effects, new_state

FX.StartActor(
    input=topic,
    initial_state=0,
    rules={(Any, Any): handle_msg},
) | run

# Send messages — the actor processes them
FX.Publish(target=topic, content='hello') | run
FX.Publish(target=topic, content='world') | run
# [0] got: hello
# [1] got: world

Refinement Types

Types are sets. Compose them with & (intersection) and | (union). Refine with predicates.

Positive = Int & (Z > 0)
SmallPositive = Positive & (Z < 100)

42 | is_a(SmallPositive) | collect    # True
-1 | is_a(SmallPositive) | collect    # False
200 | is_a(SmallPositive) | collect   # False

# Use in function dispatch
EvenInt = Int & (Z % 2 == 0)

Multiple Dispatch

Functions support multiple methods dispatched on argument types — like Julia, but in Python.

@method
def greet(x: String) -> String:
    return f"Hello, {x}!"

@method
def greet(x: Int) -> String:
    return f"Number: {x}"

greet("Alice")   # "Hello, Alice!"
greet(42)        # "Number: 42"

Core Ideas

• Data-oriented. If it can be expressed as data, it should be. Effects, errors, types, functions — all values.
• Entity identity. Entities can be unnamed, locally named, graph-local, or UID-addressed; see Entity Names, UIDs, and Local Identity.
• Managed effects. Side effects are declared as values with FX.*, then executed with | run. Pure core, imperative shell.
• Contiguous memory. Zef values live in contiguous buffers — no pointer chasing, no serialization. The in-memory format is the wire format.
• Refinement types. Types are sets. Int & (Z > 0) is a type. Compose freely.
• Content-addressable functions. Functions are identified by UID, not name. Multiple names can point to the same function. Methods are hot-reloadable.
• Actor concurrency. Actors subscribe to topics, process messages, and produce effects. State is explicit and threaded.
• Cross-language. Rust core with Python bindings (via PyO3). The same binary data flows between languages without marshaling.

Getting Started

Install from Wheel

Download the latest wheel from GitHub Releases and install:

pip install zef_core-*.whl

Build from Source

git clone [e-mail]:UlfBissbort/zef.git
cd zef

# Create a virtual environment
python3 -m venv dev_venv
source dev_venv/bin/activate

# Install maturin
cargo install maturin --locked

# Build and install (release mode recommended — debug is significantly slower at runtime)
cd zef_core
maturin develop --release --features py_bindings

Configure

Create ~/.config/zef/config.zen.py:

ET.ZefConfig(zef_source_dir='/path/to/zef', vault_=['~/zef-vault'])

Verify

>>> import zef
🚅 created shelf allocator of size 64.00 GB on thread ThreadId(1)
🌿 Zef import completed

>>> zef.__version__
'0.1.14'

Running Tests

# Rust tests
cd zef_core && cargo test

# Python tests — run all with the test runner
python run_tests.py

# Run specific tests by pattern
python run_tests.py int64 skip

# Run a single test file directly
python tests/test_int64_layout.py

See notes/How to Write Tests in Zef.md for the full testing guide.

CI/CD

python trigger_build.py          # Bump patch version and trigger build
python trigger_build.py minor    # Bump minor version
python trigger_build.py major    # Bump major version

Creates a git tag and pushes to GitHub, triggering the workflow that builds the wheel and creates a release.

Documentation

The notes/ directory is a linked knowledge base covering architecture, design decisions, and implementation details. Start with notes/Overview.md for the index.

Key guides:

• Using Zef from Python — quick-start with API examples
• Zef Architecture Overview — conceptual and source code organization
• Zef Effects System — managed side effects
• Actor FX Module — actor system architecture

The zefop-search.html file is a standalone searchable reference for all Zef operators.

Tip: The notes use [[wiki-link]] cross-references and work as an Obsidian vault — open notes/ in Obsidian for a browsable, linked view.

Status

Zef is under active development. The API is evolving. The core data structures, type system, effects system, and actor model are functional. Language support is currently Python and Rust, with TypeScript (via WebAssembly) planned.

The Rust codebase produces compiler warnings — a cleanup pass is planned once the type definitions move to the code graph and source generation is unified.

Building Wheels

Wheels are pre-built binary packages for distribution via PyPI or direct install.

The extension_module Gotcha

PyO3's extension-module feature controls how the native library links against Python. Without it, the .so gets hardlinked to the specific libpython from the build environment. The wheel then only works on the exact same Python build — install it on a different one (e.g. homebrew) and you get:

Fatal Python error: PyInterpreterState_Get: the function must be called
with the GIL held ... but the GIL is released

The extension_module feature is configured in pyproject.toml, but maturin only reads it when run from the project root. The build command must use -m to point at the sub-crate:

# ✅ Correct — reads pyproject.toml, applies extension_module feature
cd /path/to/zef
maturin build --release -m zef_core/Cargo.toml

# ❌ Wrong — skips pyproject.toml, wheel crashes on non-build Python
cd zef_core
maturin build --release

Verify the build output includes 📡 Using build options features from pyproject.toml and does not show a warning about extension-module.

Cross-Compiling for Linux

# Linux x86_64 (requires Docker or cross toolchain)
maturin build --release -m zef_core/Cargo.toml --target x86_64-unknown-linux-gnu

# Linux ARM
brew install zig
rustup target add aarch64-unknown-linux-gnu
maturin build --release -m zef_core/Cargo.toml --target aarch64-unknown-linux-gnu --zig