openforage 0.2.27

Agent-driven market strategy discovery, backtesting, and signal submission for OpenForage.

OpenForage Library

🚧 CLOSED BETA — invite code required. OpenForage is currently in closed beta. Access to the library requires an invite code. The agent will not be able to openforage register without one. Subscribe at openforage.ai to join the whitelist; invite codes are distributed to whitelisted operators as the closed beta opens. Do not start the bootstrap below until you have an invite code in hand.

Python CLI and library for the OpenForage commodity trading and analysis platform. An agent registers a wallet, runs a background search that evaluates mathematical signal expressions against historical market data, submits passing signals to the OpenForage server, and consumes callback events for failures and periodic improvement prompts.

This README is the agent's single front door. Point any modern coding agent at this file — Claude CLI, Codex CLI, Hermes, OpenClaw — and the agent can register, start a background search that downloads data while the rest of the setup happens in parallel, wire up callbacks and schedules, and begin productive iteration without opening anything else. For clean-room journey verification, use the canonical narrative source of truth, consciousness/skills/clean_room/references/clean_rooms/openforage_library/agent_user_journey.md as the testing checklist; it defers operational onboarding back to this README.

The six bundled SKILL.md files under src/openforage/skills/ are depth references; this README is the index and the active orchestration script.

---

§0 — Set a Durable Outcome Before Doing Anything Else

STOP. Before any other command, put the outcome below somewhere your runtime will preserve and re-read. If your runtime has a /goal slash command, use it. Otherwise use the runtime's native tasklist plus a durable scratchpad (HEARTBEAT.md, a pinned message, or a local GOAL.md). The important primitive is the durable outcome, not the literal /goal command.

Bootstrap OpenForage end-to-end: complete every §A task in
openforage_library/README.md, keep the background search running
healthily, register a file sink plus a scheduled_improvement callback,
poll `openforage status --json` plus `events.jsonl` until the worker is
healthy or a failure is classified, consume each improvement_prompt
event and act on it, and only stop when actual results stabilize or a
genuine blocker is reached. Read every bundled SKILL.md before changing
the run loop. Save durable discoveries to memory before exiting.

Runtime-specific ways to persist it:

• Claude CLI — create native task entries for each §A item and put the outcome in a durable scratchpad or pinned instruction. If your Claude runtime or harness extension exposes /goal, you may paste the same text there, but Claude CLI is not assumed to have a built-in /goal primitive.
• Codex CLI — use update_plan and record_tasklist for the §A items. If your Codex version exposes /goal, paste the same outcome there as an additional runtime-specific guard.
• Hermes (Nous Research runtimes that expose /goal) — paste the outcome into /goal so the runtime tick re-loads it.
• OpenClaw — persist the outcome in the Lane Queue or /goal if that runtime exposes one. Then paste the output of openforage.get_agents_md_fragment() into the workspace's AGENTS.md so the OpenClaw skill loader picks up OpenForage.

Re-read the durable outcome on every turn. There is no scenario where you can hold this in working memory across multi-turn work and remain correct.

---

§A — The Six Bootstrap Tasks (Enter These Into Your Tasklist Now)

Enter these six items into your runtime's native task tool before running any command beyond --help. Use the runtime's tasklist primitive (Claude: TaskCreate; Codex: update_plan then record_tasklist; OpenClaw / Hermes: the runtime's task store). Stable IDs are below so cross-agent handoff is unambiguous.

ID	Task	Done when
T-OF-register	Run pip install openforage, then openforage init if you need to inspect or edit .openforage/settings.yaml before registration, then openforage register --invite-code <CODE> --json. Save the printed address.	address returned, wallet.key exists under the configured data_dir, mode 0600.
T-OF-start-search-bg	Spawn nohup openforage start --json > openforage-start.log 2>&1 & in the same turn as register completes — do NOT wait for the data download.	The shell returns immediately; background.pid appears under the configured data_dir.
T-OF-callbacks-sink	While data downloads in the background, register a file sink with openforage callbacks register file --path .openforage/callback-events.jsonl. Optionally add a webhook or shell sink.	openforage callbacks list --json includes a file entry.
T-OF-schedule	Register at least one scheduled_improvement callback and wire your runtime's tick to openforage.run_due_callbacks().	openforage callbacks list --json includes a scheduled_improvement entry; your tick is scheduled.
T-OF-baseline-watch	Read §J Health Baseline. Poll openforage status --json and tail events.jsonl until you can compare against the table.	running, process_health, data_download_progress, and recent events either match the baseline or you have classified the deviation.
T-OF-improvement-loop	Consume each improvement_prompt from the sink, apply the suggested adjustment, restart the worker if needed, compare on get_search_stats() / get_yield_analysis().	One full prompt-to-comparison cycle has completed.

These six are the entire onboarding contract. Treat the list as the spec, not the README narrative. If your runtime cannot store these as first-class tasks, write them into a HEARTBEAT.md you re-read every turn.

Release-soak proof for these tasks must run on this project's owned test stack and capture the submission endpoint hit, server DB readback, terminal status transition, and exact terminal verdict. A local run, legacy shared endpoint, or log-only transcript is not acceptance evidence.

---

§B — Mental Model (One Line)

register (wallet) → search (worker discovers signals across historical eras) → submit → callbacks emit events → improvement_prompt events feed back into the agent loop.

That is the entire system. Everything else in this README is detail.

---

§B.1 — settings.yaml Contract

~/.openforage/settings.yaml is the single durable user-editable runtime configuration file for the OpenForage library. The package ships the template at openforage_library/src/openforage/settings.yaml; openforage init creates the user copy and upgrades it on later runs by preserving current keys, adding new template keys, and dropping removed keys.

Runtime loading is strict. A normal library entrypoint fails loudly when the user settings file is missing, has unknown keys, has retired keys, or is missing required keys. Use openforage init when installing or upgrading so the local file matches the packaged template.

Source order is deliberately narrow: explicit CLI/function arguments win where an API exposes them; otherwise settings.yaml is the runtime source of truth. null means "use the documented built-in behavior", such as SQLite instead of Postgres, auto-derived identity, default-on standard logging, or local-server validation disabled. It does not mean "read a legacy environment variable".

The file covers these groups:

• Local paths and service endpoints: data_dir, server_url, signal_service_url, local_server_port.
• Registration and identity: invite_code, instance_*, and author_*.
• Local persistence: db_url, db_schema, and the local-server Postgres pool setting.
• Local-server, GUI, shutdown, and logging knobs: local_server_named_universe, validation/sync toggles, gui_demo_mode, stop_wait_seconds, and standard_logging.
• Search and runtime resource knobs: algorithm, workers, memory estimate, batch size, priority, genetic parameters, and feature budget.
• Sync transport and concurrency: HTTP transport, chunk size, and max concurrency.
• Contributor stats: upload opt-out, upload URL/interval, and B2 consent references.

---

§B.2 — Logging Guide

OpenForage library logs are visible standard diagnostics. What the logs mean: they show progress through public work, including entry, exit, decision, value, and error events. Logging is on by default and default-on; routine entry, exit, decision, and value lifecycle events are DEBUG diagnostics, while error events remain visible at the default INFO threshold.

Logs go to stderr by default. The stream and destination can be supplied by the caller when configuring logging, so machine JSON on stdout stays parseable. Set verbosity with the normal Python log level; enable debug when you are actively inspecting lifecycle progress.

To opt out, run openforage init and set standard_logging: false in ~/.openforage/settings.yaml before import or process start. Do not use log suppression to hide a real failure; classify the failure from status, events, and stderr.

Internal IP and firm IP stay bounded. A cryptic token must stay cryptic, and logs should never reveal protected source detail. The coverage gate scripts/verify_openforage_library_logging_coverage.py --json checks captured library emissions, and scripts/ci_checks.sh runs that coverage gate beside the format parity check.

---

§C — Read These Skills First (Not Optional)

The six bundled SKILL.md files under src/openforage/skills/ are the source of truth for behavior. Skipping them is the single most expensive shortcut. Install/import must happen before openforage.list_skills(): the bundled skills ship inside the wheel, so discovery only works after pip install openforage has made that wheel importable in the active Python environment. Inside the agent runtime, run:

python -c "import openforage; print('\n'.join(openforage.list_skills()))"

then read every printed path before changing the run loop or invoking a CLI beyond --help / --version.

Skill	Must-read gate
openforage_quickstart	Before T-OF-register — installation, durable state directory, smallest local loop, agent handoff fields.
background_search_loops	Before T-OF-start-search-bg — worker lifecycle, restart, diagnostics, supervisor patterns.
search_settings	Before changing --algorithm / --settings-path — built-in search templates, settings validation, and safe template switches.
callback_hooks	Before T-OF-callbacks-sink — file/webhook/shell sink contracts, opt-in flags, audit log.
scheduled_improvement_loops	Before T-OF-schedule — scheduled_improvement semantics, why it does not run an optimizer, sink ordering.
agent_runtime_integration	Before T-OF-improvement-loop — durable outcome/task persistence across runtimes, polling cadence patterns, generic shell/webhook/file adapters.

Programmatic discovery:

import openforage
openforage.list_skills()        # six absolute paths to SKILL.md files

Local Settings And Data Layout

Run openforage init when you want to inspect or edit settings before registration or worker startup. It creates ~/.openforage/settings.yaml without contacting OpenForage services or starting a worker. The file is copied from the library template and merged on upgrade: new fields are added, removed template fields are removed, and current values for fields that still exist are preserved. The template is the user-editable schema; catalog-derived runtime fields such as market and named universes are intentionally not written there. The default data_dir inside that file points under the user-level OpenForage directory, so downloaded manifests, feature data, wallet state, tracking databases, background status, and worker logs later live under the configured data_dir unless you edit that settings file.

---

§D — Thirty-Line End-to-End Example

This is the smallest self-contained script that exercises the full loop on a fresh machine. Copy verbatim into a file (or pass via python -c) and run it from a working directory you control.

# openforage_smoke.py — end-to-end smoke test
import subprocess, time, json

# 1. Materialize editable settings and the durable data directory.
subprocess.run(["openforage", "init", "--json"], check=True)
# Before first run, put the operator-issued code in ~/.openforage/settings.yaml:
# invite_code: <CODE>

# 2. Register the wallet with an operator-issued invite code.
subprocess.run(["openforage", "register", "--json"], check=True)

# 3. Spawn the background search WITHOUT blocking — data downloads in the
#    background while we set up callbacks/schedules in parallel.
subprocess.Popen(
    ["openforage", "start", "--json"],
    stdout=open("openforage-start.log", "a"),
    stderr=subprocess.STDOUT,
    start_new_session=True,
)

# 4. While the worker is downloading, wire up a file sink and a
#    scheduled_improvement callback. Sink is registered FIRST so the
#    prompt has somewhere to go.
import openforage
openforage.register_callback(callback_type="file",
                             path="callback-events.jsonl")
openforage.register_callback(callback_type="scheduled_improvement",
                             interval_seconds=60,
                             prompt="Review status, recent_errors, events.jsonl, algorithm_state. Suggest one conservative adjustment.")

# 5. Poll status until the background worker is alive or a failure is visible.
for _ in range(60):
    out = subprocess.run(["openforage", "status", "--json"],
                         capture_output=True, text=True, check=True).stdout
    s = json.loads(out)
    health = s.get("process_health") or {}
    if s.get("running") and health.get("state") == "running":
        break
    if s.get("recent_errors"):
        raise RuntimeError(s["recent_errors"][-1])
    time.sleep(2)

# 6. Run due callbacks once (your runtime owns the real polling loop).
print(openforage.run_due_callbacks())

# 7. Stop cleanly before changing anything.
subprocess.run(["openforage", "stop", "--json"], check=True)

This script is intentionally agnostic of the runtime. Wrap it in whatever supervisor you already use; the orchestration belongs to the agent, not OpenForage.

---

§E — Sub-flows

The remaining sections are reference detail for the six §A tasks. Read once when you encounter them; do not pre-read everything below if the §A list and §D example already match your needs.

§E.1 Install

pip install openforage
openforage --help          # lists: init, register, start, status, stop, callbacks, serve, local
pip show openforage        # confirms the installed version
pip index versions openforage  # authoritative list of published versions
python -c "import openforage; print(openforage.list_skills())"

The wheel ships precompiled Cython .so binaries plus the six bundled SKILL.md files; openforage.list_skills() reads those files from the installed/importable wheel. A source build is only needed for library contributors (build via setuptools per pyproject.toml build-backend = "setuptools.build_meta").

Prereqs. Python 3.11 through 3.13 are supported for this release. The OpenForage wheels install on CPython 3.11, 3.12, and 3.13. The search backtester ships in the installed wheel; the first openforage start sync fetches era data, vault artifacts, per-universe vocabularies, and the encrypted shuffle seed for the selected release. You also need network access to https://api.openforage.ai (HTTPS, non-rate-limited path), a writable state directory with ≥ 30 GB free under the configured data_dir, and a stable network session for the first-register sync. Run openforage init to create ~/.openforage/settings.yaml before registration; if data_dir is not edited, the default data_dir setting is ~/.openforage/data.

Distro packages. On Debian/Ubuntu, install the target Python and matching venv package, for example python3.12 python3.12-venv. On Ubuntu releases that do not ship your target interpreter in the base archive, enable the deadsnakes PPA first. On RHEL/Fedora, use the matching python3.x package. On macOS, install a supported interpreter with Homebrew, for example brew install python@3.12. As a fallback for any host, install via pyenv.

Version confirmation. The PyPI web UI sometimes renders an older long description than what pip install will actually download. Trust pip index versions openforage or pip install openforage==<exact-version> rather than the rendered PyPI page.

§E.2 Register

Registration creates a 32-byte private key in wallet.key and authenticates it with the OpenForage server. First registration requires an operator-issued invite code. openforage register is foreground/synchronous: it returns only after wallet setup, auth, and bootstrap work either complete or fail. It uses data_dir from ~/.openforage/settings.yaml.

openforage init
openforage register --invite-code <CODE> --json

To keep the invite code in durable local configuration instead of command history, run openforage init, set invite_code: <CODE> in ~/.openforage/settings.yaml, then run:

openforage register --json

If invite_code is null, pass the code explicitly with --invite-code or register(invite_code=...).

Optional identity attribution fields also live in ~/.openforage/settings.yaml: instance_id, instance_key, instance_name, author_id, author_key, author_name, author_focus, and author_email. Leave them null for the normal auto-derived instance and wallet/default author behavior.

import openforage
openforage.register(invite_code="<CODE>")
openforage.login("<32-byte-hex-private-key>")   # returning agent with an existing key

register --json prints the agent's wallet address. There is no openforage whoami subcommand — your address is whatever register returned for this wallet.key. If you have lost the address but still have the key file, re-run openforage register --invite-code <CODE> --json against the same configured data_dir; registration is idempotent and re-prints the address.

macOS Python.org users: if register fails with an SSL cert error (SSL: CERTIFICATE_VERIFY_FAILED), see the macOS entry in Troubleshooting before retrying. The fix is one-line (SSL_CERT_FILE=$(python -m certifi)) and is not your wallet or network's fault.

Anti-pattern: do NOT add a src/ directory or a .pth file mapping src.* → openforage.*. The wheel rewrites module names internally at build time and ships only compiled .so files. If you see a namespace-mismatch error referencing src.functions.shared_utils._typed_shapes or bare simulator_engine.*, that is a wheel bug — file it at https://github.com/systematic-long-short/openforage/issues with the full traceback. Do not paper over it locally.

A user-side shim is dangerous because it can make the same compiled .so resolvable under two module names (openforage.functions._typed_shapes AND src.functions.shared_utils._typed_shapes). Cython's per-module type tables drift apart when that happens, and the worker will SIGSEGV on its first signal evaluation. Crash reports will reference cli.cpython-*.so and background.cpython-*.so, and PYTHONFAULTHANDLER=1 output will list both module names. The remedy is to delete the shim, not to add more shims.

If you have already added one: remove the src/ directory, remove any matching .pth entries (find $(python -c 'import site; print(site.getsitepackages()[0])') \ -name '*.pth' -exec grep -l '^src' {} +), exit the venv and start a fresh shell, then re-run openforage start.

State directory contents produced by register:

• wallet.key — 32-byte hex private key, mode 0600. Back this up off-host before any other command. Suggested recipe:

  DATA_DIR=$(python -c 'from openforage.settings import resolve_data_dir; print(resolve_data_dir())')
  install -m 0400 -D "$DATA_DIR/wallet.key" \
    "$HOME/.config/openforage-backup/wallet-$(date +%Y%m%dT%H%M%S).key"
  

• Vault artifacts, feature files, useful signals, found payloads, per-universe vocabularies, and shuffle_seed.enc metadata are synced on first openforage start, not on register. The search backtester itself is imported from the installed wheel. The CLI register authenticates the wallet and saves wallet.key; the era-data sync happens as part of the worker's bootstrap path inside openforage start (see §E.3 and §E.3.0 below).
• shuffle_seed.enc — encrypted seed for the obfuscation layer.
• No token cache file. Each process keeps its JWT only in AuthManager memory (_token / _token_expiry) and clears it on close or process exit. A new CLI invocation that needs auth reads wallet.key, runs register -> challenge -> EIP-191 sign -> verify, and stores the returned JWT only in that process.

Switching state directories: edit data_dir in ~/.openforage/settings.yaml before registration or worker startup. Command-line overrides are intentionally not supported.

The CLI and Python API can share the same state files only when they are the same runtime/venv/library version. The data directory is coupled to platform and versioned artifacts: manifest checks, vault artifacts, feature files, min_library_version, the local wheel cache, .sync_flock, consumer_count, and transition_lock. Non-current platform files are skipped, and era transitions may auto-update from the local wheel cache. Avoid sharing one data directory across different venvs, OpenForage versions, platforms, or agent runtimes; use one configured data_dir per runtime/venv.

→ Depth: src/openforage/skills/openforage_quickstart/SKILL.md

§E.3 Start a Search and Download Data (Spawn-and-Keep-Working)

§E.3.0 What to expect during the first download

The first openforage start (not register) triggers a roughly 20–25 GB sync of per-platform era artifacts: the vault module, features, useful_signals, found_strategies, per-universe vocabularies, and the encrypted shuffle seed. The search backtester is loaded from the installed wheel. Allow ≥ 30 GB free under the configured data_dir. The exact size for the current era becomes visible from data_download_progress.total_bytes in openforage status --json once the manifest has been written.

Indicative wall-clock:

Network class	Typical duration
Bare-metal datacenter / 1 Gbit	7–15 minutes
Residential gigabit	15–30 minutes
Residential broadband (100 Mbit)	30–90 minutes
Coffee shop / hotel wifi	1–3 hours, with retries

No cross-process resume in this release. If start exits or the worker crashes mid-download, partial .tmp files in the configured data_dir are deleted on the next sync attempt (the cleanup runs at the top of _sync_impl); restarting start re-downloads from the beginning. SHA-256 verification IS automatic within a single sync attempt (across HTTP retries inside one process). On slow networks, lean on a stable session rather than assuming you can rescue progress after a kill — and do not run the worker on a laptop that may sleep mid-bootstrap.

Settings knobs in ~/.openforage/settings.yaml (override only if you know why):

Setting	Default	When to set it
sync_http_transport	auto	Set to urllib or curl only to isolate transport-specific failures.
sync_http_chunk_bytes	67108864 (64 MB)	Lower it on unstable links to reduce retry cost per ranged download.
sync_max_concurrency	8	Lower (1-2) if the host or network cannot sustain 8 parallel TLS connections.

Watching progress. Tail events.jsonl for lifecycle events; poll openforage status --json and read data_download_progress.percent. Heartbeat freshness is NOT a reliable signal during the first download — see the "First-bootstrap heartbeat caveat" added to §J — because the worker writes one heartbeat at process start and then blocks in bootstrap until the download finishes, so process_health.state will report stale for a healthy worker for the entire download window.

The stalled_download callback event type exists in the schema (§F) but is not emitted automatically by the worker today; if you need a stalled-download alarm, emit one yourself via openforage.emit_event() from your supervisor loop.

§E.3.1 Primary recipe

Primary recipe — spawn the worker in the background and return immediately so the rest of the setup can run in parallel:

nohup openforage start --json \
                       > openforage-start.log 2>&1 &

This pattern is the whole point of T-OF-start-search-bg: the data download and historical era ingestion happen inside the spawned worker, freeing the agent to register callbacks, schedule prompts, and read skills concurrently.

openforage start returns after spawning the background process. Inside that process, the start worker path waits for the same synchronous register/bootstrap path for the configured data_dir before starting the search template, so a real search does not run ahead of wallet/auth/data bootstrap.

openforage status --json
openforage stop   --json

There is one active worker per configured data_dir. openforage start probes background.pid / status.json; if that data directory already has a running worker, it returns the current status instead of starting a second one. Treat background.pid as owned by the worker for that state directory. Separate runtimes, venvs, platforms, or OpenForage versions should use separate data directories so they do not fight over the same pid file, status files, SQLite DB, sync locks, and artifacts.

openforage stop --json sends SIGTERM, then escalates to SIGKILL only if the same PID still validates as the OpenForage worker for that configured data_dir. On a clean stop it removes or clears background.pid and writes a stopped/stale/not_running status. It does not guarantee that an in-flight submission drained before the process exited. Before mutating templates or deleting local state, confirm quiescence from openforage status --json, process_health, and the tail of events.jsonl.

Foreground smoke test (only when you are debugging the worker interactively):

openforage start --json

Python in-process equivalent (recommended when the agent runtime can call Python directly — does not need nohup):

Closed-beta load policy. Searches use all available cores by default. Tune down deliberately with n_jobs=1 when an operator or quota policy requires a single worker. In ~/.openforage/settings.yaml, process_priority accepts normal, below_normal, or idle; the default normal is a no-op that preserves current worker behavior. below_normal and idle apply only to per-candidate multiprocessing worker children, fail loudly on unsupported hosts or permission errors, and never silently fall back to normal. The public values are portable, but the current candidate multiprocessing evaluator still requires fork semantics, so this setting does not add Windows candidate multiprocessing support. process_worker_memory_gib defaults to 4.0 and caps only per-candidate child evaluator count; it does not lower public search n_jobs. The default per-candidate process budget is 300 seconds; raise it only when real simulations still exceed that budget and the overall run remains bounded. Use openforage.templates.random_weighted explicitly only for a baseline, comparison run, or fork source.

import openforage
handle = openforage.search()
status = handle.status()      # SearchStatus dataclass
# ... wait, poll, react ...
handle.stop()                 # bounded joins/flushes; see note below

SearchStatus exposes (among other fields): running, n_jobs, total_evaluated, total_found, signals_per_hour, submission_queued, submission_pending, submission_accepted, submission_rejected, errors.

SearchHandle.stop(join_timeout=2.0) flushes pending receipt finalizations, asks worker/supervisor/submission/polling threads to stop, joins them with timeouts, releases the sync manager, and drops any remaining in-memory submission queue so status can become quiescent.

Without --json, CLI result commands print human-readable labelled summaries for terminal use. Use openforage status --follow for a live refreshing worker summary plus new lifecycle events. Use status --json when another program needs the machine-readable contract.

openforage status --json / status.json expose a background status envelope designed for scheduled prompts. Read these keys explicitly:

• pid, running
• process_health.state, process_health.heartbeat_age_seconds
• data_download_progress.phase, data_download_progress.completed_bytes, data_download_progress.total_bytes, data_download_progress.percent
• signals_found, signals_submitted
• estimated_earnings, actual_earnings
• recent_errors
• algorithm_state.algorithm, algorithm_state.worker_template
• top-level algorithm, worker_template
• path fields pid_file, heartbeat_file, status_json_file, event_jsonl_file, worker_diagnostic_jsonl_file, plus the same values under paths

Do not expect the background status envelope to contain the in-process SearchStatus metrics such as total_evaluated, total_found, or signals_per_hour. Those are available from handle.status() when you own the Python SearchHandle; the CLI background loop exposes liveness, bootstrap errors, paths, and persisted counters.

The worker writes the following into the configured data_dir:

File	Purpose
status.json	Latest parseable status snapshot.
heartbeat.json	Process liveness.
background.pid	Worker process id.
worker-diagnostics.jsonl	Worker diagnostic JSONL with stdout/stderr, native fatal, sync, escaped raw_text, and standard_line evidence.
events.jsonl	Append-only JSON Lines lifecycle stream.
tracking.db	SQLite store for signals and submissions.
callbacks.json	Registered callback definitions plus audit log.

Tail events.jsonl for live progress. Read it as JSON Lines (one object per line). Do not edit it in place while a worker is running.

Switching to PostgreSQL instead of the default SQLite store:

# ~/.openforage/settings.yaml
db_url: postgresql://user:pass@host/dbname

Once set, every persistence surface in the library — the local-server portal, the GUI simulator, the background worker, evaluation tracking, submission queues, and openforage status — routes through Postgres. Schemas are bootstrapped idempotently on first start; switching back to sqlite is as simple as setting db_url: null. psycopg2-binary must be installed in the same environment.

Production-style Postgres databases often revoke CREATE on public. For those databases, create a dedicated SDK schema and point the library at it:

# ~/.openforage/settings.yaml
db_schema: openforage_agent_state
# or encode the same setting in db_url:
db_url: postgresql://user:pass@host/dbname?options=-csearch_path%3Dopenforage_agent_state

The connected role must have USAGE and CREATE on the active schema because the SDK runs idempotent CREATE TABLE IF NOT EXISTS / ALTER TABLE ... ADD COLUMN IF NOT EXISTS at startup. After dropping and recreating a Postgres database, recreate the dedicated schema before restarting:

CREATE SCHEMA IF NOT EXISTS openforage_agent_state AUTHORIZATION openforage_agent;
ALTER ROLE openforage_agent IN DATABASE openforage
  SET search_path TO openforage_agent_state;

Do not grant CREATE on public in a shared database just to satisfy the SDK bootstrap.

Migrating existing data sqlite → Postgres

If you already have local history in tracking.db and want to carry it forward into a freshly-configured Postgres database, stop the worker first (openforage stop) and then run:

openforage db migrate \
    --from sqlite                       # or sqlite:/explicit/path/tracking.db
    --to "postgresql://user:pass@host/dbname" \
    --archive-source

The default mode refuses to run if the destination already has rows. Use --resume to skip rows that conflict on the primary key when restarting a partial migration (does not update mutable rows). Use --replace to TRUNCATE every destination table and reload from sqlite — the right choice when you want a fresh sync after continued sqlite use.

--archive-source moves the sqlite file (and its -wal/-shm siblings) to tracking.db.migrated-<UTC-timestamp> only after verification (per-table row count + SHA-256 over sorted row content) passes. A failed verification rolls back the Postgres side and leaves the sqlite source untouched.

Cross-agent handoff payload: the exact configured data_dir, the last openforage status --json payload, the last few lines of events.jsonl, and whether the worker is expected to be running.

→ Depth: src/openforage/skills/background_search_loops/SKILL.md

§E.4 Hooks and Schedules

In OpenForage:

• Hooks = callbacks. Four types: file, webhook, shell, scheduled_improvement.
• Schedules = scheduled_improvement callbacks driven by openforage.run_due_callbacks().

There is no built-in scheduler daemon. The agent runtime owns the polling loop that calls run_due_callbacks() (cron, APScheduler, asyncio, or any tick the agent already runs). This is intentional — see §F for the event schemas a sink will actually receive.

§E.4a Register an output sink first

A scheduled prompt with no sink registered has nowhere to go. The file sink is the safe default:

openforage callbacks register file \
  --path .openforage/callback-events.jsonl

§E.4b Optional — webhook sink

import openforage
openforage.register_callback(
    callback_type="webhook",
    url="https://example.com/openforage",        # HTTPS only; transport is validated
    headers={"Authorization": "Bearer ***"},
)

§E.4c Optional — shell sink

Shell callbacks execute a fixed argv with the event JSON delivered on stdin. The library requires explicit opt-in:

openforage.register_callback(
    callback_type="shell",
    command=["/usr/local/bin/notify"],
    allow_command_execution=True,                # required; absence raises ValueError
)

To observe a shell sink safely before letting it run on real events, register it under a wrapper that logs argv + stdin to a file first:

cat > /tmp/of-trace-sink.sh <<'SH'
#!/usr/bin/env bash
exec >> /tmp/of-shell-sink.log 2>&1
echo "--- $(date -u +%FT%TZ) ---"
echo "argv: $*"
cat   # stdin = event JSON
SH
chmod +x /tmp/of-trace-sink.sh
# then register command=["/tmp/of-trace-sink.sh"]

§E.4d Schedule periodic improvement prompts

openforage callbacks register scheduled_improvement \
  --interval-seconds 3600 \
  --prompt "Review status, recent_errors, events.jsonl, and algorithm_state. Suggest one conservative algorithm adjustment." \
  --json

openforage.register_callback(
    callback_type="scheduled_improvement",
    interval_seconds=3600,
    prompt=(
        "Review status, recent_errors, events.jsonl, and algorithm_state. "
        "Suggest one conservative algorithm adjustment."
    ),
)

scheduled_improvement does not run an optimizer. It emits an improvement_prompt event to your registered sinks at each due interval. Your agent runtime decides what to do with the prompt. There is no hard registry max/rate limit: each enabled scheduled callback that is due emits one improvement_prompt per run_due_callbacks() call.

§E.4e Wire run_due_callbacks() into your runtime's loop

result = openforage.run_due_callbacks()
# result["events_emitted"]       — number of due prompts dispatched
# result["callbacks_dispatched"] — number of sinks that received them

Call this on whatever cadence your runtime already uses. Recommended patterns per runtime:

• Claude CLI: a ScheduleWakeup every 60–90s during active work, or a CronCreate(* * * * *) for unattended runs.
• Codex CLI: schedule a one-line tick in update_plan that fires the call hourly, plus an event-driven tick on each user turn.
• Hermes: invoke from the runtime tick cycle or /goal loop if configured.
• OpenClaw: invoke from the Lane Queue's periodic skill, or from a cron entry on the agent host.

Dispatch is synchronous and serial. Each due scheduled_improvement event is sent to all enabled non-scheduled sinks (file, webhook, shell) before the next due scheduled callback is processed. Effective rate is controlled by the callbacks' interval_seconds values and by how often your runtime calls run_due_callbacks().

§E.4f Manage the registry

openforage callbacks list   --json
openforage callbacks emit   --event-type failure --severity warning
openforage callbacks remove --callback-id <ID>

→ Depth: src/openforage/skills/callback_hooks/SKILL.md → Depth: src/openforage/skills/scheduled_improvement_loops/SKILL.md → Depth: src/openforage/skills/agent_runtime_integration/SKILL.md

§E.5 Iterate

Three iteration loops, in order of cadence:

(a) Within a search — if you own a Python SearchHandle, poll handle.status() until total_found and signals_per_hour stabilize. For the CLI background worker, use openforage status --json for liveness/errors and events.jsonl for lifecycle milestones. Stop the worker, persist findings, decide next move.

(b) On the workflow — consume improvement_prompt events from your callback sink. Modify the template or algorithm, stop the worker, restart with the new configuration. To clear local state between runs:

openforage.delete_evaluations()        # worker must be stopped first

(c) On the algorithm — fork random_weighted, validate locally, and compare against baseline via the analytics surface in §L.

Caveat on numbers: status fields like estimated_earnings and actual_earnings, plus submission payment / payment_usdc values, are server/API-reported off-chain metadata and local ledger observations, not client-signed settlement. The library does not sign on-chain transactions, pay gas, or move funds. Do not promise earnings to upstream callers.

→ Depth: src/openforage/skills/scheduled_improvement_loops/SKILL.md (the random_weighted fork review checklist)

---

§F — Event Payload Schemas

All five event types share the same envelope (from src/openforage/callbacks.py). The payload dict's content depends on the event type and on who emits the event.

Common envelope:

{
  "event_type": "<failure | degraded_submission | stalled_download | broken_algorithm | improvement_prompt>",
  "severity":   "<info | warning | error | critical>",
  "timestamp":  "30-06-26 12:00:00.123",
  "payload":    { "...": "..." }
}

Per-type payload conventions (what the sink will actually see). The failure / degraded_submission / stalled_download / broken_algorithm payloads are emitted by the worker or by your own emit_event() calls; the shapes below are the canonical fields used across the codebase.

failure — a worker step raised an exception.

{
  "event_type": "failure",
  "severity":   "warning",
  "timestamp":  "30-06-26 12:00:00.123",
  "payload":    { "reason": "string", "where": "string", "detail": "optional string" }
}

degraded_submission — a signal cleared evaluation but the server rejected the submission.

{
  "event_type": "degraded_submission",
  "severity":   "warning",
  "timestamp":  "30-06-26 12:00:00.123",
  "payload":    { "signal_id": "string", "server_reason": "string" }
}

stalled_download — era artifact sync has not progressed within the expected window.

{
  "event_type": "stalled_download",
  "severity":   "warning",
  "timestamp":  "30-06-26 12:00:00.123",
  "payload":    { "era_number": "integer-or-null", "since_seconds": 600 }
}

broken_algorithm — the chosen algorithm/template raised on its own inputs (not on data) — typically means the template is misconfigured.

{
  "event_type": "broken_algorithm",
  "severity":   "error",
  "timestamp":  "30-06-26 12:00:00.123",
  "payload":    { "algorithm": "random_weighted", "reason": "string" }
}

improvement_prompt — emitted by run_due_callbacks() when a scheduled_improvement callback's interval has elapsed.

{
  "event_type": "improvement_prompt",
  "severity":   "info",
  "timestamp":  "30-06-26 12:00:00.123",
  "payload":    { "prompt": "Review status, recent_errors, ..." }
}

Sink contracts:

• file sinks append one JSON object per line to the configured path.
• webhook sinks POST the envelope as JSON, with any headers registered on the callback. Bearer transport is validated by _transport_security.validate_bearer_transport_url.
• shell sinks invoke the fixed argv and write the JSON envelope on stdin (no environment variables, no shell expansion).

---

§G — Template Anatomy: random_weighted

openforage.templates.random_weighted is the explicit baseline algorithm. It is a Python generator function (defined at src/openforage/search_templates/random_weighted.py:399) that yields candidate expressions for the worker to evaluate.

Conceptually, an expression is a tree:

   function_root            (chosen by yield-weighted sampling from
        |                    safe rotation functions)
        |
   function_intermediates    (depth uniformly in [2, max_depth])
        |
   feature_leaves            (chosen by yield-weighted sampling)

Key behaviors:

• Cold start (no history): uniform sampling across safe rotation functions and valid features.
• Warm phase: weight w_i = base_weight * (1 + yield_rate_i), normalized. Higher-yield functions and features sampled more often.
• Depth: uniformly random in [2, max_depth], where max_depth is read from the current era config (see Hard Rules below).
• Node budget: capped at the current era's max_node_count and clipped to fit the historical buffer's buffer_duration_minutes.
• Recursive depth limit: 128. If the node budget cannot fit a rotation root, the template logs a disable reason and exits.

Reproducibility note: random_weighted has no public seed CLI knob. It uses Python's module-level random internally (random.randint, random.choice, random.random). For reproducible runs, control the random seed in the Python process or test harness, or save the generated expressions/configs you want to compare. Do not look for a CLI seed flag.

Forking procedure (copy this template, change one knob at a time):

import openforage
from openforage.search_templates import random_weighted as base

def my_template(ctx):
    # Example fork: bias toward depth 3 instead of [2, max_depth]
    for expr in base(ctx):
        if expr.depth() == 3:
            yield expr

handle = openforage.search(my_template, n_jobs=1)

When forking the template, your filter MUST reject any candidate with expr.depth() > current_era.max_depth. The runtime exposes the era-derived limit; do not copy a numeric depth cap into a fork.

Before promoting a forked template, run a baseline (random_weighted) and your fork against the same era/universe and compare on get_yield_analysis()'s lifetime_yield_rate and get_search_stats()'s composite_score columns — see §L.

→ Depth: src/openforage/skills/scheduled_improvement_loops/SKILL.md

Hard Rules

• Signal compute-graph depth MUST NOT exceed the current era's max_depth. This applies to random_weighted, every fork of random_weighted, every custom template, and every directly submitted candidate. Agents are required to self-reject any candidate with expr.depth() > current_era.max_depth before evaluation or submission. The cap applies to the signal compute graph only — feature depth is independent and not subject to this rule.

---

§H — Algorithm Catalog

As of this version, the stable algorithm templates documented for first-run agents are genetic and random_weighted. genetic is the default when --algorithm is omitted. random_weighted remains an explicit baseline, comparison target, and fork source. The CLI --algorithm argument for both openforage start and the internal _worker command uses argparse choices for those names, so unsupported strings are rejected before the worker starts. Forking your own algorithm follows the §G procedure through the Python API.

Programmatic enumeration:

import openforage
STABLE_ALGORITHM_TEMPLATES = ("genetic", "random_weighted")
{name: getattr(openforage.templates, name) for name in STABLE_ALGORITHM_TEMPLATES}

Do not discover algorithm templates by listing every public callable on openforage.templates; that module also exposes classes, analytics helpers, and search-management functions. If the stable template tuple above grows beyond these names, the new template should ship with its own SKILL.md entry — open one of the existing SKILL.md files in src/openforage/skills/ to copy the structure.

---

§I — Cost, Mode, and Real-Money Boundary

• Registration is free. openforage register creates a local private key and authenticates a JWT against api.openforage.ai. No payment is required and no on-chain transaction is signed.
• Submission is free. Submitting a passing signal to the OpenForage server does not require gas, fees, or a deposit. Earnings are awarded asynchronously and reported via the actual_earnings status field; actual_earnings and submission payment values are server/API-reported off-chain metadata and local ledger observations, not guarantees.
• There is no documented testnet, dev, or paper-trading mode in this version of the library. openforage serve runs a local FastAPI portal (§K) but is not an isolated paper-trading endpoint. Treat register and signal submission as touching production.
• Real-money boundary. An agent may proceed without asking only for no-cost/dev/testnet/paper-trading work and bounded real exposure up to and including USD 100 nominal exposure (<= USD 100). Ask before any higher credible economic risk unless the user has explicitly authorized it. The library itself does not sign on-chain transactions, move funds, or pay gas, but downstream automations (custody, withdrawals, derivative exposure built on top of returned signals) can. Keep credentials off-host and audit any wallet that touches mainnet.

---

§J — Health Baseline

Use the checks below to distinguish "worker is alive" from "bootstrap or supervision is stalled." Numbers vary with host, network, and era — treat them as classification hints, not SLAs.

data_download_progress is a dict. Use data_download_progress.percent for numeric comparisons; the same object also carries phase, completed_bytes, and total_bytes. In the CLI background status envelope this field is a status slot, not the in-process search-rate surface. The in-process metrics total_evaluated, total_found, and signals_per_hour belong to SearchStatus from handle.status().

Time since openforage start	running	process_health.state	data_download_progress.percent	events.jsonl / recent_errors
0–30 seconds	usually true	running	0.0 unless a worker writes progress	start and heartbeat events should appear
1 minute	true	running with fresh heartbeat age	source-reported percent if available	expect register, search_start, or a worker_error
5 minutes	true or classified failure	running, stale, or not_running	source-reported percent if available	no fresh heartbeat or a nonempty recent_errors list requires classification
1 hour+	true for unattended run	heartbeat age remains bounded	source-reported percent if available	compare analytics / in-process SearchStatus if you need search-rate metrics

First-bootstrap heartbeat caveat. The worker writes a single heartbeat at process start and then blocks in registration + era-data download (see §E.3.0) for the entire download window. During this period — which can be 1–3 hours on slow networks — process_health.state will report stale and process_health.heartbeat_age_seconds will keep growing, even though the worker is healthy. The reliable liveness signal during bootstrap is running: true (pid still alive) combined with absence of recent_errors and absence of a worker_error event in events.jsonl. Treat process_health.state: stale plus running: true plus no errors plus growing data_download_progress.percent as a healthy bootstrap, not a stall. Do not restart the worker on stale-heartbeat alone during the first download — you will only re-trigger the full re-download (no cross-process resume).

Classification:

• running: false with process_health.state: "not_running" right after start → worker never stayed up; inspect diagnostic JSONL worker-diagnostics.jsonl and events.jsonl.
• running: false and process_state: "stale" within 30 seconds of openforage start, no register event in events.jsonl, and diagnostic JSONL ending without a Python traceback (or ending in a system signal trace) → worker exited via an uncatchable signal (SIGSEGV / SIGBUS). The two known causes are (a) a src/ shim duplicate-load (see §E.2 anti-pattern callout) and (b) a per-era binary built for a different Python ABI (see §E.1 Prereqs). Inspect the diagnostic JSONL tail: if raw_text ends mid-import of a .so file, ABI mismatch is the likely cause; if it ends after the first register event, shim duplicate-load is the likely cause. SIGSEGV does not emit a failure callback event — the signal is uncatchable in Python.
• process_health.state: "stale" or heartbeat age grows far beyond your polling cadence → pid file or worker supervision is stale; inspect background.pid, diagnostic JSONL, and recent events before restarting. First-bootstrap exception: if no register event has appeared yet in events.jsonl and data_download_progress.percent is still climbing, this is the expected stale-during-bootstrap state — keep waiting (see the caveat above).
• Nonempty recent_errors or a worker_error event → classify the bootstrap/search failure before restarting. Network or auth failures usually surface here.
• data_download_progress.phase: "blocked" with a recent error such as checksum_repair_exhausted → the worker is alive but the current sync cannot complete from the advertised public bytes. Inspect the named file in recent_errors, verify the manifest SHA against the public object and backing object store, and fix the data-publish / cache path before restarting. A fresh heartbeat does not make this state healthy.
• Need search-rate metrics such as total_evaluated or signals_per_hour → use a Python SearchHandle and handle.status(), or inspect analytics after the worker has produced local evaluation data. Do not read those fields from openforage status --json.

If you do not have your own baseline yet, run the §D example for an hour, save its final status.json plus the tail of events.jsonl, and use that as your local norm.

---

§K — openforage serve and openforage local

Two advanced subcommands that fall outside the standard agent flow. Documented here so they do not appear unexplained in --help.

• openforage serve [--port 17320] [--host 127.0.0.1] [--log-level info] — Runs a local OpenForage portal server (FastAPI + sqlite) for interactive inspection. Not required for agent onboarding; use it when you want a browser-side view of local state.
• openforage local install | uninstall | status | start [--foreground] | stop — Manages an OS-level user supervisor service so the background worker is restarted automatically across host reboots. Useful for long-running unattended agents; not required for the §A bootstrap. If you install a supervisor service, remember it is the agent's responsibility to clean it up via openforage local uninstall before deleting the worktree.

Local-runner supervisor stdout/stderr support evidence is written to diagnostic JSONL paths under {data_dir}/local_runner/, primarily diagnostics.jsonl for supervisor evidence and serve-diagnostics.jsonl for the served process.

§K.1 Running in Docker / Kubernetes

openforage start's PID-file supervision is not namespace-aware: _probe_process() calls os.kill(pid, 0) against the PID written to background.pid. In a container whose PID namespace differs from the supervisor's, that probe is meaningless. Symptoms: openforage status reports stale or unknown state, openforage stop fails to terminate the worker, or background.pid is a single-digit number that maps to a kernel thread on the host.

The clean fix is to run the worker as the container's main process, not via openforage start. _worker is the underlying foreground command:

FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends tini \
    && rm -rf /var/lib/apt/lists/*
RUN pip install "openforage>=0.1.4"
WORKDIR /state
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["openforage", "_worker", "--settings-path", "/state/settings.yaml"]

Notes:

• Mount /state from a persistent volume sized for the first-register download (§E.3.0, ≥ 30 GB).
• tini is optional but recommended; it ensures orphan reaping if the worker spawns children.
• openforage status and openforage stop only work meaningfully from inside the same PID namespace as the worker (i.e., docker exec into the same container). Calling them from a sidecar container will read background.pid correctly but the os.kill(pid, 0) probe will return misleading results.
• Searches use all available cores by default inside containers. Add an explicit --n-jobs 1 only when an operator or quota policy requires tuning down to a single worker.

For Kubernetes, the same _worker command works as the container's command field; pair with a livenessProbe that reads status.json from the mounted volume rather than relying on PID-file checks.

---

§X — Contributor Web Local Runner Setup

Use this flow when an agent is connecting the local OpenForage library to https://openforage.ai/app/contributor. The commands are idempotent and require no sudo.

1. Install or upgrade the library in the active venv:

   pip install --upgrade openforage
   

2. Register the user-level supervisor:

   openforage local install --json
   

   The command registers a LaunchAgent on macOS, a systemd --user service on Linux, or a Scheduled Task on Windows.

3. Verify supervisor health within 30 seconds:

   curl -sS http://127.0.0.1:17319/healthz
   

   Expected shape:

   {"ok": true}
   

4. Verify simulation server health within 30 seconds:

   curl -sS http://127.0.0.1:17320/healthz
   

   Expected shape:

   {"ok": true, "version": "0.1.8", "port": 17320, "local_capability_token": "..."}
   

   If version is older than 0.1.8, repeat step 1 with:

   pip install --upgrade "openforage>=0.1.8"
   

If step 3 or 4 still fails after openforage local install --json exits successfully, run the foreground fallback:

openforage local start --foreground

Then re-check both /healthz endpoints. If either endpoint still fails, halt and report:

• OS name and version.
• Last 50 diagnostic JSONL rows from {data_dir}/local_runner/diagnostics.jsonl.
• The failing curl command and full response.
• Next user action: re-run openforage local install --json with a writable configured data_dir, then check OS service-manager availability.

---

§L — Analytics Return Shapes

Each openforage.get_search_* function returns a typed dataclass from native_sources/src/openforage/search_analytics.pyx. Field cheat-sheet:

get_search_stats(...)        # → SearchStatsResult(
    # rows=[StatsRow(group_key, windows=[WindowStats(window, sharpe, returns, turnover,
    #                                                drawdown, correlation, yield_rate,
    #                                                submittable_yield, composite_score,
    #                                                evaluation_count)], evaluation_count)],
    # aggregate_by, eval_pool, filters_applied, total_evaluations, total_groups, groups_shown)

get_suggestions(...)         # → SuggestionsResult(focus=[StatsRow], avoid=[StatsRow],
                              #                    aggregate_by, scoring_formula)

get_search_patterns(...)     # → SearchPatternsResult(
    # depth_distribution=[DepthBucket(depth, total, passed, yield_rate)],
    # family_yield_rates=[FamilyYield(function_family, total, passed, yield_rate)],
    # total_evaluations, total_passed)

get_search_velocity(...)     # → SearchVelocityResult(
    # windows=[VelocityWindow(window_minutes, evaluations_per_hour, passes_per_hour,
    #                         total_evaluations, total_passed)], trend)

get_yield_analysis(...)      # → YieldAnalysisResult(lifetime_yield_rate, recent_yield_rate,
                              #                       trend, recent_window_minutes,
                              #                       total_evaluations, total_passed,
                              #                       diminishing_returns)

get_correlation_patterns(...)# → CorrelationPatternsResult(
    # pairs=[FunctionPairPattern(function_a, function_b, total_cooccurrences,
    #                             passed_cooccurrences, lift)],
    # total_passing_evaluations, total_evaluations)

Auxiliary listings:

openforage.get_available_eras()         # → list[str]
openforage.get_available_universes()    # → list[str]
openforage.get_available_aggregations() # → list[openforage.search_analytics.AggregateBy]

Compare baseline vs fork on cold and warm distributions before promoting any algorithm change (§G).

---

§M — Autoresearch and Log Reconstruction Guide

Use this runtime-agnostic loop for recurring improvement work:

1. Schedule a recurring prompt with scheduled_improvement, or call openforage.run_due_callbacks() from your runtime's own scheduler.
2. Edit or rewrite a readable algorithm file, usually a fork of openforage.search_templates.random_weighted or genetic.
3. Run it with openforage.search(settings_path="~/.openforage/settings.yaml") or openforage start --settings-path ~/.openforage/settings.yaml; the settings file can set default_algorithm to the fork's .py path and process_priority to normal, below_normal, or idle, plus process_worker_memory_gib for per-candidate child evaluator memory estimates.
4. Measure search statistics with get_search_stats() or get_yield_analysis() before and after the run.
5. Iterate only when the measured stats change; record the seed, settings file, algorithm file, and evidence artifact.

Capture the diagnostic JSONL output for the run. To reconstruct one log chain, filter rows by one group_id, sort the matching records by numeric log_id values from the log_id field, then read the ordered standard_line messages as the causal chain. Keep each row's escaped raw_text with the run evidence so another agent can replay the reconstruction.

---

Troubleshooting

• openforage: command not found — the entry point did not install into the active venv. Reinstall in the right venv, or invoke python -m openforage.cli instead.
• macOS — SSL: CERTIFICATE_VERIFY_FAILED during register / first download. Python.org installers on macOS do not configure the OpenSSL trust store. Point Python at certifi's CA bundle before re-running:

  pip install certifi
  export SSL_CERT_FILE="$(python -m certifi)"
  openforage register --json
  

  Test whether you need this:

  python -c "import ssl, urllib.request; urllib.request.urlopen('https://api.openforage.ai', timeout=10).close()"
  

  If it raises SSL: CERTIFICATE_VERIFY_FAILED, apply the workaround above. Homebrew Python (brew install python@3.12) usually has a populated trust store and does not need this. Note: REQUESTS_CA_BUNDLE does not affect OpenForage's stdlib urllib path; only SSL_CERT_FILE works for OpenForage today.
• undefined symbol: _Py_Version when loading native wheel or vault extensions. Recreate the venv on Python 3.12 and re-run openforage start:

  deactivate || true
  DATA_DIR=$(python -c 'from openforage.settings import resolve_data_dir; print(resolve_data_dir())')
  rm -rf .venv "$DATA_DIR"/vault_module-*.so \
         "$DATA_DIR"/*.tmp
  python3.12 -m venv .venv
  source .venv/bin/activate
  pip install --upgrade pip
  pip install openforage
  openforage register --json
  openforage start --json
  

  Cleaning leftover vault .so and .tmp files is intentional; stale native artifacts from the wrong Python ABI must be removed before the next start sync.
• openforage start fails before signal evaluation begins. The vault artifacts and era data sync during the worker's bootstrap (the first openforage start), not during openforage register. If diagnostic JSONL raw_text shows a network or HTTP 401 error during sync, fix connectivity to https://api.openforage.ai and re-run openforage start. To start the sync from a clean state, remove vault_module-*.so and any *.tmp files from the configured data_dir first. There is no offline mode and no cross-process resume — keep the session stable through the first sync (see §E.3.0).
• Worker dies with SIGSEGV in cli.cpython-*.so or background.cpython-*.so after loading per-era binaries. Two known causes:
  • Shim duplicate-load. A user-side src/ directory or .pth mapping causes the same compiled .so to load under two module names (openforage.functions._typed_shapes AND src.functions.shared_utils._typed_shapes). See the anti-pattern callout in §E.2.
  • Python ABI mismatch. The per-era binary was built for a different CPython version than the running interpreter. See the _Py_Version entry above. Distinguish them by looking at diagnostic JSONL raw_text: a SIGSEGV mid-import (before a register event in events.jsonl) points to ABI; a SIGSEGV after the register event points to the shim duplicate-load. SIGSEGV is uncatchable, so no failure callback event will fire — read diagnostic JSONL and events.jsonl directly. Set PYTHONFAULTHANDLER=1 before openforage start if you want a Python-side native traceback printed at crash time.
• JWT refresh fails repeatedly — the in-memory JWT auto-refreshes at ~80% of its 24h lifetime. Persistent refresh failure usually means api.openforage.ai is unreachable from this host. Validate connectivity (curl -sI https://api.openforage.ai) before assuming the wallet itself is broken.
• Search starts then exits immediately, no SIGSEGV. Era data, vault artifacts, or the shuffle seed are missing, wrong-platform, or partially synced. Remove vault_module-*.so and any matching *.tmp from the configured data_dir and re-run openforage start (NOT register — register does not sync era artifacts). If the same exit pattern repeats, see the SIGSEGV entry above to rule out a native crash.
• "PostgreSQL did not become ready" without postgresql-client installed — the library falls back to a socket probe automatically. If you still see this, set db_url explicitly in settings.yaml or set it to null to use SQLite.
• python3 -m venv fails with "ensurepip is not available". The distro's venv module is not installed. Install it before retrying:
  • Debian/Ubuntu: sudo apt install python3.12-venv (on Ubuntu 22.04 and earlier, enable the deadsnakes PPA first; see §E.1 Distro packages).
  • RHEL/Fedora: sudo dnf install python3.12.
  • Or pin a specific Python with pyenv install 3.12.7 && pyenv local 3.12.7.
• PyPI page shows an older version than pip installs. PyPI's rendered description can lag behind the latest release. Trust pip index versions openforage or pip install openforage==<exact-version> rather than the rendered PyPI page.
• Scheduled callbacks fire but nothing visible happens — register at least one file, webhook, or shell sink. scheduled_improvement on its own has no output channel.
• stop reports that the worker survived SIGKILL — check background.pid, process_health, and OS process state for that configured data_dir. Manual kill and pid-file cleanup should be a last resort because normal openforage stop --json already performs the bounded safe escalation.
• Shell sink should be observed before going live — register it via the dry-run wrapper from §E.4c first; verify /tmp/of-shell-sink.log shows the expected argv and JSON before promoting it.

---

Concept Glossary

• Signal — a mathematical formula that predicts which instruments will perform well or poorly. Must clear quality thresholds (Sharpe, annualized return, turnover) to pass evaluation.
• Functions — about 382 building-block operations across 6 families: numerical, longitudinal, latitudinal, collection, comparison, event.
• Features — about 302 input data columns; each is a time-series matrix across instruments.
• Expressions — tree-shaped combinations of functions over features (function root, function intermediates, feature leaves).
• Obfuscation — function and feature names are hashed and the data ordering is shuffled. Work with patterns, not identities.

For deeper concept treatment see src/openforage/skills/openforage_quickstart/SKILL.md.

---

Skills Catalog

Skill	Path	Use when
openforage-quickstart	src/openforage/skills/openforage_quickstart/SKILL.md	First-run install, register, durable state directory, smallest local loop.
background-search-loops	src/openforage/skills/background_search_loops/SKILL.md	Run, supervise, resume, stop, diagnose background workers.
search-settings	src/openforage/skills/search_settings/SKILL.md	Choose and validate built-in search templates and settings files.
callback-hooks	src/openforage/skills/callback_hooks/SKILL.md	Register, list, emit, remove file/webhook/shell callbacks.
scheduled-improvement-loops	src/openforage/skills/scheduled_improvement_loops/SKILL.md	Periodic improvement_prompt events via run_due_callbacks().
agent-runtime-integration	src/openforage/skills/agent_runtime_integration/SKILL.md	Bridge OpenForage to a Claude / Codex / Hermes / OpenClaw runtime.