elastik 8.2.1

AuditeDB: the db that listens. Python client for the Elastik L5 Engine.

AuditeDB Python SDK

Python client and launcher for AuditeDB, powered by the Elastik L5 Engine.

The engine is small on purpose: the library has five verbs, while the HTTP adapter maps them to PUT, GET, HEAD, POST, DELETE, and LISTEN. The SDK gives that wire surface a Python shape without turning the core into a framework.

The beginner surface in one breath:

import secrets
import elastik

e = elastik.start(key=secrets.token_hex(32), write_token="write-token")
e.put("note", "hello")
print(e.get("note"))       # b"hello"
print(e.get_text("note"))  # hello
print(e.head("note"))      # lowercased HTTP headers
elastik.stop()

No hidden object model: put() replaces bytes, post() appends bytes, get() returns bytes, and head() returns headers.

API At A Glance

Want to...	Use
Store/load bytes	e.put(path, data), e.get(path)
Store/load text or JSON	e.put_text / e.get_text, e.put_json / e.get_json
Use dict style	e["note"] = b"hello", del e["note"], "note" in e
Stream a big read	e.open(path) (read-only file-like, Range-backed)
Use pathlib style	e / "home" / "note" (WorldRef)
Browse virtual directories	e.ls("home"), e.tree("home"), ref.iterdir()
Move/delete prefixes	e.mv(src, dst), e.rm(prefix, recursive=True)
Read several paths	e.get_many([...]) (concurrent HTTP requests)
Watch changes	@elastik.listen(pattern) + elastik.run(e)
Inspect metadata	e.head, e.checksum, e.preview, e.diff
Use scratch space	with e.tmp() as path:
Drop to raw HTTP	e.request(method, path, headers=...)
Send one UDP-curl packet	CoapClient, python -m elastik coap

Prefer e.put(...) instance methods in libraries, tests, and long-running tools where client lifecycle should be explicit. Use module-level elastik.put(...) in scripts and notebooks with exactly one core per process.

Path Contracts

AuditeDB apps can be coordinated by path names instead of API schemas.

// Frontend writes input and listens for output.
await fetch("/home/order/123", {
  method: "PUT",
  body: JSON.stringify({ sku: "tea", qty: 2 }),
  headers: { "Content-Type": "application/json" },
});

new EventSource("/listen/home/receipt/*");

# Business worker owns the workflow.
import elastik

@elastik.listen("/home/order/*")
def on_order(body, path, e):
    order_id = path.rsplit("/", 1)[-1]
    e.put_json(f"/home/receipt/{order_id}", {"status": "accepted"})

elastik.run()

The shared contract is only:

/home/order/{id}
/home/receipt/{id}

Use curl to inspect either side of the handoff:

curl.exe http://127.0.0.1:3105/home/order/123
curl.exe http://127.0.0.1:3105/home/receipt/123

In a source checkout, runnable examples live in sdk/examples/:

python sdk/examples/01_basic.py
python sdk/examples/02_listener.py
python sdk/examples/03_metadata_and_etag.py

Install

py -m pip install elastik

The package name stays elastik for compatibility. AuditeDB is the product; the Python package talks to the Elastik L5 Engine.

The package ships a platform-specific elastik-core binary in elastik/_bin/. No compile-on-install.

PyPI wheels are the normal install path. If your platform does not have a wheel yet, build the Rust core from source and use an editable checkout.

Starting A Core

You have two normal choices.

1. Start from Python

Use this in scripts, tests, notebooks, and local tools:

import secrets
import elastik

e = elastik.start(
    key=secrets.token_hex(32),   # required HMAC key for the audit chain
    read_token="read-token",     # optional: omit for public reads
    write_token="write-token",   # optional: ordinary PUT/POST
    approve_token="admin-token", # optional: DELETE and protected namespaces
)

read-token, write-token, admin-token, and dev-hmac-key in these examples are local placeholders only. The engine does not ship with built-in credentials. Use fresh per-deployment secrets outside throwaway demos.

Python kwargs use underscores (read_token). CLI flags use hyphens (--read-token).

2. Start from a terminal

Use this when you want a long-running local service:

py -m elastik run --key dev-hmac-key --read-token read-token --write-token write-token --approve-token admin-token

Then connect from another process:

from elastik import Elastik

e = Elastik("http://127.0.0.1:3105", bearer_token="write-token")

Module-level elastik.put/get/... calls require either a prior elastik.start(...) or explicit environment like ELASTIK_URL and ELASTIK_WRITE_TOKEN. They do not silently assume that an unknown process on 127.0.0.1:3105 is yours.

Tokens

• read_token: gates GET, HEAD, /listen/*, and read-only /proc/* requests such as /proc/worlds, /proc/du, /proc/df, /proc/pool, and /proc/audit/<world>/verify. OPTIONS is policy-free.
• write_token: ordinary write token for PUT and POST.
• approve_token: admin token for DELETE and protected namespaces (etc/, lib/, boot/, usr/, var/log/). Non-log var/ uses the ordinary write token.

If read_token is omitted, reads are public. If write_token is omitted, ordinary writes are disabled. If approve_token is omitted, destructive/admin operations are disabled.

Migration note: ELASTIK_TOKEN was the old write-token name. It still works as a temporary fallback when ELASTIK_WRITE_TOKEN is unset, but the SDK warns so you can rename it.

CoAP / UDP Helper

The binary can expose an opt-in CoAP/UDP surface when ELASTIK_COAP_PORT is set. The SDK gives humans a small translator so nobody has to hand-type CoAP bytes:

python -m elastik coap put 127.0.0.1 5683 home/sensor/temp "23.5" --token write-token
python -m elastik coap get 127.0.0.1 5683 home/sensor/temp --token read-token

Or keep the endpoint once:

from elastik import CoapClient

c = CoapClient("127.0.0.1", 5683, token="write-token")
c.put("home/sensor/temp", "23.5").raise_for_status()
print(c.get("home/sensor/temp").payload)

This is not a full RFC 7252 CoAP stack. It is a CoAP-shaped UDP adapter: one datagram in, one datagram out, GET/PUT, Uri-Path, Content-Format, payload bytes, and response codes. The SDK only advertises text/plain and application/octet-stream; JSON/CBOR content formats are intentionally not part of the no-JSON CoAP surface. It does not implement retransmission, dedup, Observe, Block-Wise transfer, DTLS/OSCORE, .well-known/core, multicast discovery, Max-Age, or strict critical-option handling.

AuditeDB private option 65001 carries the raw auth token, like a UDP-shaped Authorization: Bearer .... It is not encryption. Use a CoAP gateway at the edge if you need full CoAP behavior; use HTTP for reliability or large bodies. Bodies near 1 KiB or above should use HTTP: the SDK enforces a 1152-byte datagram limit and rejects oversize PUTs with ValueError immediately.

Paths

"foo" and "/foo" both mean /home/foo.

Explicit namespaces are allowed when you want their storage policy:

• /home/*: durable SQLite storage.
• /tmp/*, /dev/*, /sys/*: memory-backed storage.
• /proc/version, /proc/worlds, /proc/du, /proc/df, /proc/pool: core introspection endpoints.

Namespace roots like /home, /tmp, /lib, /etc, /var/log, and /proc/* internals are reserved. Store application data under a child path such as /home/myapp/data.

Concrete mapping:

Input path	Stored/read path
"note"	/home/note
"/note"	/home/note
"tmp/scratch"	/tmp/scratch
"/tmp/scratch"	/tmp/scratch
"proc/worlds"	/proc/worlds
"proc/pool"	/proc/pool
"proc/audit/home/note/verify"	/proc/audit/home/note/verify
"/proc/unknown"	rejected

list_paths() is the preferred name. list_keys() and the older list_worlds() name remain as aliases; all three read /proc/worlds.

print(e.get_text("/proc/version"))  # core version string
print(e.list_paths())               # /proc/worlds, one path per line

Metadata

Standard representation metadata has named arguments:

e.put(
    "report.pdf",
    pdf_bytes,
    content_type="application/pdf",
    content_disposition='attachment; filename="report.pdf"',
    cache_control="max-age=60",
)

Extra keyword arguments become plain X-Meta-* headers:

e.put("note", "hello", project="demo")
# X-Meta-Project is sent on the wire. Whether it round-trips depends on
# the core's persist policy (see below). On a default v7.2+ core the
# next line raises KeyError unless you start the core with
# ELASTIK_PERSIST_HEADERS=x-meta-* (or pass persist_allow=... to a
# FakeElastik fixture).
assert e.head("note")["x-meta-project"] == "demo"

Any safe response header that does not have a named argument can be sent through headers=:

e.put(
    "logo.png",
    png_bytes,
    content_type="image/png",
    headers={"Access-Control-Allow-Origin": "*"},
)
# CORS family is in the built-in default-allow set; no env config needed.
assert e.head("logo.png")["access-control-allow-origin"] == "*"

Persist policy (v7.2+)

The core's persist decision is four layers:

1. Hard deny (hardcoded): credentials, hop-by-hop, distributed-tracing context (traceparent / x-b3-* / x-amzn-* / cf-*), IP-leak headers, HTTP/2+3 pseudo-headers, and core-owned response headers (ETag, Content-Length, Link, Allow, X-Request-Id, ...). Operators cannot turn this off.
2. User deny (ELASTIK_DENY_HEADERS): operator subtracts from the allow layers below — beats both layer 3 (default allow) and layer 4 (user allow). Affects future writes only; already-persisted headers round-trip until the world is re-PUT.
3. Default allow (hardcoded): standard representation headers (Content-Disposition / Content-Encoding / Content-Language / Content-MD5 / Cache-Control / Expires / full CORS family / CSP / X-Frame-Options / Permissions-Policy / COEP/COOP/CORP / Referrer-Policy / X-Robots-Tag).
4. User allow (ELASTIK_PERSIST_HEADERS): operator opts in custom names like X-Author or X-Meta-*.

Default empty layer 4 means no custom headers round-trip without configuration. To restore the v7.1 default for X-Meta-*:

export ELASTIK_PERSIST_HEADERS=x-meta-*

Content-Type is special: stored as the representation media type, not as generic persisted metadata. When X-Meta-* is allowlisted, durable worlds audit it as persisted representation metadata (meta_sha256 / event_headers).

The SDK also refuses wire-level headers that urllib must compute itself: Content-Length, Transfer-Encoding, Host, Connection, Keep-Alive, TE, Trailer, Upgrade, and HTTP2-Settings. Passing those through headers= raises ValueError instead of letting a bad length hang the request.

Authorization is allowed as an explicit escape hatch. If you pass it in headers=, it takes precedence over the client's bearer_token.

Bytes, Text, JSON

get() is byte-exact:

e.put("x", "hello")
assert e.get("x") == b"hello"

get() raises ElastikError(404, ...) when a path is missing. None only means 304 Not Modified from an if_none_match cache check; it never means "missing".

Use helpers when you want decoding:

e.put_text("note", "hello")
e.put_json("config", {"debug": True})
e.get_text("x")          # str
e.get_json("config")     # parsed JSON

head() returns a typed header dict (WorldMeta) for editor help:

meta = e.head("x")
print(meta["etag"])
print(meta["content-type"])

Common WorldMeta keys are optional HTTP headers: etag, content-type, content-length, content-encoding, content-language, content-disposition, cache-control, accept-ranges, and link.

Conditional And Partial Reads

The SDK exposes common HTTP controls directly:

etag = e.head("config")["etag"]
e.put("config", b"new", if_match=etag)     # optimistic update
e.put("lock", b"mine", create_only=True)   # If-None-Match: *
chunk = e.get("big.bin", range=(0, 1023))  # Range: bytes=0-1023

if_none_match is for ETag strings only. Use create_only=True for If-None-Match: *.

For anything not sugared, use the raw HTTP escape hatch:

r = e.request("OPTIONS", "note")
print(r.status, r.headers, r.body)

Python Ergonomics

The elastik-core binary exposes HTTP; the SDK adds small Python-shaped conveniences without hiding the wire.

e["note"] = "hello"             # PUT /home/note
assert e["note"] == b"hello"    # GET /home/note
assert "note" in e              # HEAD /home/note
assert e.exists("note")
assert e.sizeof("note") == 5
e.copy("note", "note-copy")      # GET + HEAD + PUT
del e["note"]                   # DELETE /home/note

Stored paths are canonicalized to <namespace>/<rest> with no leading slash: e["src"], e["/src"], and e["home/src"] all index the same path. Iteration returns that canonical core form from /proc/worlds, such as home/src or tmp/scratch.

The core store is flat: home/sensor/kitchen/temp is one key, not three real directories. The SDK gives Python users a virtual hierarchy by splitting paths on /, like pathlib, os, and shutil:

e.ls("home/sensor")              # immediate children; virtual dirs end in /
e.ls("home/sensor", depth=-1)    # all descendants
print(e.tree("home"))
e.du("home/sensor")              # {path: content_length}
e.mv("home/draft", "home/final") # copy+delete; refuses overwrite by default
e.rm("home/old", recursive=True) # refuses "" or namespace roots without force=True

mv() is copy+delete, not an atomic filesystem rename. Partial failures can leave source and destination paths side by side. rm("home", recursive=True) and rm("", recursive=True) are guarded footguns; pass force=True only when you really mean to delete a namespace or the whole store.

copy() buffers the source body in Python memory. That is fine for ordinary objects; for huge blobs, use a streaming tool or curl pipeline.

Elastik implements collections.abc.MutableMapping[str, bytes], so update(), pop(k, default), setdefault(), keys(), values(), and items() work too. Each mapping operation is still one HTTP request; there is no hidden transaction or batch endpoint.

More stdlib-shaped helpers are thin wrappers over HTTP:

e.get_cached("note")                  # GET + If-None-Match cache
e.checksum("note")                    # HEAD -> ETag
e.is_audited("note")                  # True if ETag is hmac-backed
e.verify("note")                      # HEAD /proc/audit/home/note/verify
e.diff("note", "new text")            # local unified diff
e.preview("note", max_bytes=512)      # Range GET + text preview
e.put_gzip("log.gz", "hello")         # Content-Encoding: gzip
e.put_csv("table.csv", [["t", "v"]])  # text/csv
e.put_struct("dev/s0", ">ff", 1.0, 2.0)
e.get_many(["a", "b"])                # concurrent GETs, no batch endpoint

is_audited() is only a storage-mode check. It tells you whether the current ETag is HMAC-backed, not whether the full audit chain has been replayed. verify() asks the core to replay the durable audit chain via HEAD /proc/audit/{path}/verify. It returns True only for 200 OK with X-Audit-Valid: true; memory worlds (204 No Content) and broken chains (409 Conflict) return False.

open(path, "rb") returns a read-only file-like object backed by Range GETs:

with e.open("report.pdf") as f:
    header = f.read(8)
    f.seek(1024)
    chunk = f.read(512)

Pathlib-shaped references are available when they make code clearer:

report = e / "home" / "reports" / "q1"
report.write("revenue up")
print(report.read_text())
print(report.stat()["etag"])

for child in (e / "home" / "reports").iterdir():
    print(child.name, child.suffix)

for pdf in (e / "home").rglob("*.pdf"):
    print(pdf.path)

Temporary paths are just /tmp/* paths with best-effort cleanup:

with e.tmp("scratch") as path:
    e.put(path, "working...")

Errors have subclasses when you want precise handling:

try:
    e.get("missing")
except elastik.NotFound:
    print("not there")
except elastik.PreconditionFailed:
    print("etag changed")

For bug reports and shell sanity checks:

import elastik
print(elastik.__version__)
elastik.show_config()

If you look directly inside the durable data/ directory, names are percent encoded because the core stores a flat keyspace safely on Windows and POSIX:

home/note.txt  -> data/home%2Fnote%2Etxt/universe.db

Use the ops helpers when you need to translate that layout:

python -m elastik decode-path "home%2Fnote%2Etxt"
python -m elastik ls-data .\data

Listening For Changes

@listen is optional. Do not call elastik.run() unless you registered at least one handler.

import elastik

e = elastik.start(key="dev-key", write_token="write-token")

@elastik.listen("/home/inbox/*")
def on_inbox(body, path, meta, e):
    if b"urgent" in body:
        e.put("/home/alerts/latest", body)

elastik.run(e)

Handler rules:

• The first positional argument is always body.
• Extra context is injected by name: path, etag, pattern, meta, e, method, and event.
• world is still accepted as a compatibility alias for path.
• You can do normal Python side effects inside the handler.
• Advanced users may return Reply, Archive, MoveTo, or Drop action objects, but they are not required.

Use clear_routes() or unlisten(pattern) in tests or notebooks to reset handler state. Registering the same pattern twice raises unless you use listen(pattern, replace=True).

run() retries forever by default and logs failures to stderr. For supervised processes, prefer elastik.run(e, reconnect=False) and let your supervisor restart the process. For demos/tests, max_events=1 runs until one matching event is handled.

Environment Loading

import elastik loads a local .env once and fills only missing environment variables. Existing process env wins. Set ELASTIK_NO_DOTENV=1 before import to disable this, or call elastik.load_dotenv(path) explicitly when you want manual control.

Advanced Helpers

These are exported but not part of the beginner path:

• request(): raw HTTP escape hatch.
• binary_info(), is_running(), default_url(): launcher diagnostics.
• TrustedShellPool: warm local shell process pool for trusted @listen handlers. It can execute arbitrary commands; do not feed it untrusted input.
• MoveTo, Reply, Archive, Drop, Action, Ctx: optional reactor action vocabulary.

Stateless By Default

The SDK intentionally uses one-shot stdlib HTTP requests by default: no requests, no urllib3, no connection pool, no hidden keep-alive state.

That is slower than a tuned keep-alive client, but it is boring and hard to leak. High-frequency callers can use curl, ab -k, http.client, or a custom transport when they have measured a real bottleneck.

Testing Without A Core

For unit tests that only need SDK behavior, use the in-memory fake:

from elastik import FakeElastik

e = FakeElastik()
e.put("note", "hello")
assert e.get_text("note") == "hello"

FakeElastik is not a protocol test. Use the black-box tests or a real elastik.start(...) when you need wire-level HTTP behavior.

Source Checkout

git clone https://github.com/rangersui/AuditeDB
cd AuditeDB
python -m pip install -e .\sdk
python -m elastik run --key dev-hmac-key --read-token read-token --write-token write-token --approve-token admin-token

For the full project README, see:

https://github.com/rangersui/AuditeDB