nousergon-lib
Part of Crucible — a Nous Ergon product: a harness for rigorous AI/ML experiments in finance, an equity research-and-trading system instrumented end-to-end. Repo and S3 names use the underlying project codename
alpha-engine.
Shared utility library used by all 6 modules of Nous Ergon. Cross-cutting concerns only — logging, freshness checks, trading-calendar arithmetic, ArcticDB helpers, agent-decision capture, LLM cost tracking. No proprietary trading logic, no model weights, no agent prompts.
The lib's job is to keep the same code from being maintained six times.
Install
# requirements.txt
nousergon-lib @ git+https://github.com/nousergon/nousergon-lib@v0.4.0
Tagged releases: v0.1.0, v0.2.0, v0.3.0, v0.4.0, etc. Consumers pin to a specific tag. Breaking changes bump the minor version while Alpha Engine is in pre-1.0.
# With optional extras
pip install "nousergon-lib[arcticdb] @ git+https://github.com/nousergon/nousergon-lib@v0.4.0"
| Extra | Pulls in | When you need it |
|---|---|---|
[arcticdb] |
arcticdb, pandas |
Anything that calls check_arcticdb_fresh or the ArcticDB read/write helpers |
[flow_doctor] |
flow-doctor |
Logging integration that escalates ERROR-level events to flow-doctor |
[rag] |
psycopg2-binary, pgvector, numpy |
The rag submodule — Neon pgvector RAG retrieval/ingestion |
[rag-parquet] |
pandas, pyarrow |
rag.parquet_mirror — S3 parquet batch-tier mirror written at ingest time |
[rag-local-ann] |
pandas, pyarrow, hnswlib |
rag.local_ann — in-process HNSW index over the parquet tier for batch consumers (no Neon egress) |
[dev] |
pytest, lint tooling |
Local development |
Modules
logging — structured logging + flow-doctor attach
Replaces the near-identical log_config.py copies that used to live in alpha-engine-data and alpha-engine-executor. Consumers call setup_logging once at process startup:
from nousergon_lib.logging import setup_logging
setup_logging("data-collector", flow_doctor_yaml="/path/to/flow-doctor.yaml")
- Text mode by default; JSON via
ALPHA_ENGINE_JSON_LOGS=1 - Flow-doctor is default-on (0.58.0+): it attaches an ERROR-level handler whenever a
flow_doctor_yamlis provided (requires[flow_doctor]extra).FLOW_DOCTOR_DISABLED=1is the kill switch; test runs auto-disable unlessFLOW_DOCTOR_ALLOW_IN_TESTS=1. Deployed runtimes (Lambda, orALPHA_ENGINE_DEPLOYED=1) fail loud on misconfig; local dev/CI logs a WARNING and skips. - Wrap an entrypoint in
guard_entrypoint()(context manager) or@monitor_handler(Lambda decorator) so an uncaughtraiseis also captured — not justlogger.error()records. Both no-op when flow-doctor is inactive.
preflight — fail-fast connectivity + freshness checks
Runs at the top of every entrypoint, before any real work starts. Primitives live on BasePreflight; each consumer subclasses and overrides run():
from nousergon_lib.preflight import BasePreflight
class DataPreflight(BasePreflight):
def __init__(self, bucket, mode):
super().__init__(bucket)
self.mode = mode
def run(self):
self.check_env_vars("AWS_REGION")
if self.mode == "phase1":
self.check_env_vars("FRED_API_KEY", "POLYGON_API_KEY")
self.check_s3_bucket()
if self.mode == "daily":
self.check_arcticdb_fresh("universe", "SPY", max_stale_days=4)
Failed checks raise RuntimeError with an explanatory message. Consumers catch nothing — the raise propagates up through main() → non-zero exit → Step Function HandleFailure → flow-doctor notification. The point is to fail before paying for any LLM calls or downstream work.
arcticdb — read/write helpers + symbol enumeration
Wrappers around the ArcticDB Python client. Standardizes the URI format, library naming, and read paths so each consumer doesn't reinvent the connection logic.
dates — trading-day arithmetic
now_dual() returns a (calendar_date, trading_day) pair following the rule trading_day = last_closed_trading_day(now). Strictly backward-looking; never ahead. session_for_timestamp(ts) resolves any timestamp to its trading session. Used at every artifact-write site to prevent calendar/trading-day drift between modules.
trading_calendar — NYSE holiday detection
Pure-Python NYSE calendar through 2030. No pandas-market-calendars dependency.
artifact_freshness — absence-driven S3 artifact monitoring substrate
The lib-side piece of the artifact-freshness-monitor arc closing the silent absence-of-artifact bug class. SF Catch, flow-doctor, and substrate-health-check are all event-driven (failure → alert); this module is the substrate for the absence-driven complement (silence → alert).
from datetime import datetime, date, timezone
from nousergon_lib.artifact_freshness import (
ArtifactSpec, check_freshness, resolve_dedup_key,
)
from nousergon_lib.alerts import publish
spec = ArtifactSpec(
artifact_id="backtest_pit_parity",
s3_bucket="alpha-engine-research",
s3_key_template="backtest/{date}/pit_parity.json",
cadence="saturday_sf",
sla_minutes_after_cron=180,
severity="warning",
owner_repo="alpha-engine-backtester",
created_at=date(2026, 5, 27),
)
result = check_freshness(s3_client, spec, datetime.now(timezone.utc))
if result.state in ("missing", "stale", "probe_failed"):
publish(
f"[{result.state}] {spec.artifact_id}: {result.reason}",
severity=spec.severity,
dedup_key=resolve_dedup_key(spec, datetime.now(timezone.utc)),
)
Pure function — check_freshness(s3_client, spec, now) returns a CheckResult with no side effects beyond the injected s3_client.head_object. NYSE-holiday-aware (Memorial Day Monday weekday-SF cron returns state="fresh" with a holiday reason). Recovery-substitution-aware (canonical 404 + recovery-key fresh ⇒ state="fresh" with recovery_substituted=True). Grace-period gate for newly-onboarded specs (default 2 cycles).
The freshness-monitor Lambda (alpha-engine-data/lambdas/freshness_monitor/, ships in a follow-up PR) walks the alpha-engine-config/private-docs/ARTIFACT_REGISTRY.yaml SoT, calls this substrate per row, and routes via nousergon_lib.alerts.publish with the resolved dedup key.
artifact_resolution — windowed S3 artifact resolution (consumer side)
The CONSUMER half of the artifact-resilience principle (alpha-engine-config#1190): resolve a dated S3 artifact to the freshest instance within a trailing window, never to one exact run-date key — so a partial / retried / off-cycle Saturday SF run still resolves instead of reading N/A. The "clean Saturday run" is redefined as "the freshest required artifacts exist within their freshness window," however many partial runs produced them.
from nousergon_lib.artifact_resolution import get_json_windowed
# Walks back day-by-day from run_date (default 10d); skips corrupt mid-writes,
# returns the freshest GOOD instance + its real provenance date.
doc, src_date, age_days, key = get_json_windowed(
s3_client, "alpha-engine-research", "backtest/{date}/e2e_lift.json", run_date,
)
if doc is None:
... # genuinely N/A past the window — never silently graded "today"
Single source of truth, consolidating ≥4 independent reimplementations of this scan (crucible-evaluator get_json_windowed, crucible-executor read_signals_with_fallback + eod_reconcile, crucible-predictor/backtester signal-fallback). resolve_windowed_artifact is the generic HEAD-only resolver (returns a ResolvedArtifact with the freshest key) and supports a latest-pointer-first fast path. Fail-loud: a missing key keeps the scan walking back; a corrupt half-written JSON candidate is skipped to the last good one; any other S3 error (auth / throttle / wrong-bucket) is raised. Mirror of the same "freshest within a window, never the exact key" rule the artifact_freshness monitor enforces on the alerting side.
decision_set — the fleet's shared decision-set resolver
signals/{date}/signals.json::universe is a sizing envelope (one row per name on the whole ~900-name scanner board, so the executor can size and exit anything it might hold) — it is not a scope, but it is the easiest ticker list in the system to reach, so per-ticker stages kept reading it as if it were one (alpha-engine-config#5809). The correct artifact is universe_membership/{date}/membership.json, published by the scanner with named cuts (scanner_candidates@60, attractiveness_top_20@25, ...). This module resolves any named cut by an O(1) latest.json pointer read, unions in held positions (a position needs evidence whether or not it ranks this cycle), and drops non-equity identifiers (e.g. Metron's Treasury CUSIPs) with a strict ticker regex:
from nousergon_lib.decision_set import load_decision_set, DecisionSetUnavailable, CUT_SCANNER_CANDIDATES
try:
result = load_decision_set(cut=CUT_SCANNER_CANDIDATES, bucket="alpha-engine-research")
except DecisionSetUnavailable:
... # raised, never widened to signals.json::universe — see module docstring
tickers = result["tickers"]
Lifted from nousergon-data/rag/pipelines/_rag_scope.py (config-I5700, the first of two independent implementations — the second is alpha-engine-predictor/inference/stages/load_universe.py) and generalized from one hardcoded cut to any named cut. Fail-loud, no fallback: DecisionSetUnavailable is raised on a missing membership artifact or an empty/absent named cut — there is deliberately no fallback to signals.json::universe, since that fallback is the defect this module exists to remove.
decision_capture — agent decision audit logger
Captures every agent decision as a structured artifact: prompt metadata (id + version), input snapshot, agent output, and cost. Each decision becomes replayable, auditable, and attributable to a specific prompt revision. Backbone of the Phase 2 measurement substrate.
cost — LLM cost tracking
Token-aware cost computation following Anthropic's prompt-caching semantics (cache-write vs cache-read pricing). Used by every LLM call site to attach a cost_usd to its output.
cost_gate — pre-merge cost gate (console entry: nousergon-cost-gate)
Prices the diff: refuses a change that adds spend with no budget line. Grades four
classes over the lines a branch ADDS — SDK client constructions, entitlement strings,
new always-on CloudFormation resources, and new schedules (retry bound + target billing
service, read structurally from the head tree; GitHub Actions crons on private repos
only, where minutes bill). Every other cost control is detective-after-spend; this is
the preventive one, so the exit codes are the contract: 0 clean, 1 findings,
2 could not grade, with the cause named — 2 is never folded into 1, and
--warn-only downgrades findings without ever downgrading 2. It deepens a shallow CI
clone itself rather than exiting 128 on an unreachable merge base.
The approved set comes from the prefix-only cost SSoT packaged with this library
(cost_gate/data/) and from nowhere else, so widening the gate means editing a source
document in a reviewed PR — there is no second registry. That document is generated by
allowlist from a private source and carries service names, billing prefixes and no
amounts; tests/test_cost_gate_no_leak.py asserts that over the serialised bytes and
over a closed key set. --ssot / --resource-map substitute another copy, which the
tests prove grades identically.
Shipped here rather than as a reusable workflow because a public repo structurally
cannot call a private one, and a cross-repo fetch would need a credential in every
caller: every repo runs the same grader from the library it already pins. Stdlib +
pyyaml only.
nousergon-cost-gate --base origin/main --head HEAD --repo-visibility public
agent_schemas — canonical LLM-output Pydantic schemas
Shared contract surface for the 14 LLM-output classes used in with_structured_output(...) calls across the research pipeline (sector quant + qual + peer review, macro economist + critic, held-stock thesis update, CIO, eval-judge rubric). Lives here so downstream tooling — replay harness in alpha-engine-backtester, future cheap-model-concordance signals — can validate against the canonical contract without a heavy cross-repo dep on research.
from nousergon_lib.agent_schemas import (
QuantAnalystOutput,
JointFinalizationOutput,
CIORawOutput,
HeldThesisUpdateLLMOutput,
resolve_schema_for_agent,
)
# Dispatch by captured agent_id (e.g. "sector_quant:technology" → QuantAnalystOutput)
schema = resolve_schema_for_agent(agent_id)
SCHEMA_BY_AGENT_ID_BASE covers the 6 canonical agent families: sector_quant, sector_qual, sector_peer_review, macro_economist, ic_cio, thesis_update. Validators that defend observed LLM failure modes (sector-modifier clamp, JSON-string-as-list parser, min_length=1 on CIO decisions) move with their classes.
pillars — canonical 6-pillar attractiveness scoring shapes
Pydantic shapes for the institutional / SOTA refactor of research-module composite scoring — replaces the opaque quant_score + qual_score two-bucket model with a canonical 6-pillar decomposition: Quality / Value / Momentum / Growth / Stewardship / Defensiveness. Pillar set is the AQR Style Premia / Morningstar Economic Moat / Greenblatt / Piotroski / Fama-French / Asness "QMJ" consensus.
from nousergon_lib.pillars import (
PILLARS,
MoatAssessment,
PillarSubscore,
QualitativePillarAssessment,
)
# Qual Analyst emits via with_structured_output(QualitativePillarAssessment).
# Each of the 6 PillarSubscore fields carries 0-100 + confidence + evidence;
# the Quality pillar additionally carries a structured MoatAssessment
# (Morningstar wide/narrow/none + 6-archetype primary moat type + trend) —
# the qualitative core of Quality, persisted per ticker for time-series
# trend tracking.
Each PillarSubscore decomposes into optional quant_component (from the factor substrate) + qual_component (from the agent rubric) for traceability through the composite scoring layer. Catalyst is preserved as an orthogonal catalyst_horizon_modulation: int ∈ [-20, 20] (a horizon shift on near-term attractiveness), not a 7th pillar weight.
Origin: 2026-05-20 attractiveness-pillars-260520 plan-doc arc. Phase 1 (this module) ships the schema layer; Phases 2-7 wire it through alpha-engine-research, alpha-engine-data, alpha-engine-backtester, and alpha-engine-dashboard.
rag — semantic retrieval over SEC filings, transcripts, and theses
Neon pgvector backbone shared by alpha-engine-research (qual analyst's query_filings tool) and alpha-engine-data (weekly RAGIngestion step). Re-exports a small surface — retrieve, ingest_document, document_exists, embed_texts, get_connection, is_available — and ships the canonical schema.sql as package data.
from nousergon_lib.rag import retrieve
results = retrieve(
query="competitive risks and market position",
tickers=["AAPL"],
doc_types=["10-K", "10-Q", "earnings_transcript"],
top_k=8,
)
Requires the [rag] extra. Embeddings are Voyage voyage-3-lite (512d); the database backend is Neon Postgres with pgvector + HNSW indexes.
S3 parquet batch tier + local ANN (config#2958)
ingest_document mirrors every document + its chunks to s3://alpha-engine-research/rag/parquet/doc_type=.../filed_date=.../<doc_id>.parquet (Hive-partitioned, append-only, best-effort relative to the Neon insert — see parquet_mirror.py's module docstring). Batch consumers (filing_change_detection, evals, backtests) read this partition set instead of querying Neon, so a full-corpus batch scan costs zero Neon egress:
from nousergon_lib.rag.local_ann import list_parquet_keys, load_corpus_dataframe, build_local_ann_index
keys = list_parquet_keys(doc_type="10-K")
corpus = load_corpus_dataframe(keys) # plain DataFrame — groupby/aggregate consumers stop here
index = build_local_ann_index(corpus) # HNSW cosine index — nearest-neighbor consumers use this
matches = index.query(query_embedding, k=8)
Requires [rag-parquet] (mirror write path) and/or [rag-local-ann] (batch read + ANN path) in addition to [rag]. Neon remains the sole live low-latency retrieval path (pgvector HNSW + tsvector hybrid) — this tier is additive, not a replacement.
ssm_dispatcher — SSM send-command + poll chokepoint
Canonical Python primitive for the run_ssm bash helper that previously appeared as a ~54-line mirror across each dispatcher script that drives a spot instance over the SSM transport. The pre-lift shape — base64-wrap the script body, aws ssm send-command --document-name AWS-RunShellScript, loop on get-command-invocation, stream the StandardOutputContent delta, propagate the inner exit — now lives in one place where the polling cadence, error-class handling, and S3 output-key layout match across every consumer.
python -m nousergon_lib.ssm_dispatcher run \
--instance-id "$INSTANCE_ID" \
--description "bootstrap" \
--timeout 3600 \
--output-bucket "$S3_BUCKET" \
--output-key-prefix "${S3_STAGING_PREFIX}/ssm-output" \
--region "$AWS_REGION" \
--script-stdin <<'BOOTSTRAP'
set -eo pipefail
export HOME=/home/ec2-user AWS_REGION=us-east-1
# ...the script body the SSM target will execute...
BOOTSTRAP
Exit 0 on Success; exit 1 on any terminal non-Success status, send-command failure, or unrecoverable poll failure; exit 2 on bad CLI input. InvocationDoesNotExist during the first 60s after SendCommand counts as a registration race and keeps polling — closes the 2026-05-23 Saturday SF substrate weakness at the chokepoint rather than per-SF-JSON Retry block.
ssm_log_capture — SSM-step log capture + S3 ship-on-exit chokepoint
Pairs with ssm_dispatcher on the SSM target side. The dispatcher script tells the target instance to invoke python -m nousergon_lib.ssm_log_capture run --slug X --log /var/log/X.log -- bash <launcher>; the target wrapper tees the launcher's stdout/stderr to a local log file and to its own stdout (so the SSM StandardOutputContent channel still surfaces output to the dispatcher), then on exit ships the full local log to s3://alpha-engine-research/_ssm_logs/{slug}/{date}/{host}-{time}.log regardless of the inner exit code. Replaces the inline trap 'aws s3 cp ...' EXIT pattern that broke under ASL States.Array escape semantics (2026-05-22 Friday-PM dry-pass catch).
ec2_spot — capacity-resilient spot-launch chokepoint
Rotates across (instance_type × subnet) combinations on InsufficientInstanceCapacity / InsufficientHostCapacity / Unsupported / InvalidAvailabilityZone / SpotMaxPriceTooLow; non-capacity errors raise immediately. CLI exit 64 distinguishes capacity exhaustion from generic failure. Replaces the hardcoded single-subnet + single-instance-type launch pattern that mirrored across each dispatcher; landed 2026-05-22 after the third-recurrence-in-a-month spot-launch fragility.
spot_dispatch — shared Lambda-dispatcher primitives (concurrency lock, launch-with-fallback, terminate-on-failure)
Extracted (config#2106) from two independent implementations of the same three primitives — scheduled-groom-dispatcher/index.py (config#1432) and ci-watch-dispatcher/index.py (config#2001), both in nousergon-data — after a third dispatcher (sf-watch-spot-dispatcher) was about to become a third independent copy. launch_with_fallback() wraps ec2_spot.launch() with the spot-then-on-demand-on-SpotCapacityExhausted fallback (or immediate on-demand via force_on_demand). wait_ssm_online() blocks until an instance is running and its SSM agent registers Online. send_async_command() fires a detached AWS-RunShellScript SSM command with a CloudWatch-logged output config. running_instance_ids(tag_name, discriminator_tags) is a fail-safe-open (never blocks a launch on a broken check) tag-based concurrency lock — callers supply whatever discriminator tags define "the same unit of work" for their dispatcher (a groom tier, a (repo, sha) pair, an (cadence, pipeline) pair). terminate_on_failure() is a best-effort, never-raising cleanup for a box whose post-launch steps failed before its own on-box watchdog was armed. Deliberately no opinion on raise-vs-return-clean failure posture — callers wrap these (which raise) in their own try/except if they need a fail-soft synchronous contract, exactly as ci-watch-dispatcher already does around ec2_spot's exceptions.
locks — producer-side writer locks via S3 conditional PUT
universe_writer_lock(writer_id, ttl_seconds=3600) context manager that uses PutObject(IfNoneMatch="*") to claim a single-writer lease on s3://alpha-engine-research/locks/universe-writer.lock. The first writer's conditional PUT succeeds; subsequent writers get LockHeldByAnotherWriterError carrying the live LockHolder body (writer_id + started_at + ttl_epoch + hostname + pid) for operator diagnostics. Soft-TTL self-recovery deletes-and-re-acquires when the on-disk lock's ttl_epoch has elapsed; the operator-side S3 lifecycle on locks/ (expires-after-days=1) is the hard backstop. Release on context exit is best-effort and never masks an inner exception. Closes the producer-side half of the same single-writer-per-resource invariant the SF MutualExclusionGuard (DynamoDB-side) covers at the Step Function entry point; the lib lift is the chokepoint that picks up the third adopter for free (predictor weight-promote, backfill loops, etc.).
dispatch_lease — per-lane singleton dispatch lease (async-hold, TTL + forced override)
acquire_lease(lock_key, owner_id=..., ttl_seconds=..., force=False) / release_lease(lock_key) (alpha-engine-config-I6460, groom-sweep-policy.md §5.9). Same S3 PutObject(IfNoneMatch="*") + soft-TTL mechanism as locks.universe_writer_lock, exposed as an explicit acquire/release pair instead of a context manager because a scheduled dispatcher's actual unit of work (a spot box it launches and detaches from) outlives the process holding the lease — there is no live with block to hold open for the box's multi-hour run. acquire_lease never retries against a live, unexpired conflicting holder and never blocks: a caller that cannot take the lease gets back the conflicting LeaseHolder and yields immediately (§5.9: "does not queue behind the holder and does not run beside it"). A stale lease (elapsed ttl_epoch) self-recovers with no operator action. force=True skips the TTL/liveness check entirely for a caller that has already independently confirmed (via live infra state, never inferred by this module) that the recorded holder is dead — e.g. a bounded Step-Functions relaunch after a spot-reclaim check — and still reports a conflict rather than looping if a third party wins the race in the same instant. First adopter: scheduled-groom-dispatcher's per-tier launch guard.
config — experiment-package-first config resolution
resolve_experiment_config(subdir, filename, *, repo_root, extra_fallbacks=(), ...) is the canonical lift of the experiment-package-first config-resolution pattern mirrored inline across the five Alpha Engine entrypoints (alpha-engine-research/config.py::_find_config, alpha-engine-data/weekly_collector.py::load_config, crucible-executor/executor/config_loader.py, crucible-backtester/pipeline_common.py::load_config, and crucible-predictor/config.py). It searches, in order: the experiment-package copy <root>/experiments/$ALPHA_ENGINE_EXPERIMENT_ID/<subdir>/<file> (default experiment reference, baked in) for each config root, then the legacy top-level <root>/<subdir>/<file>, then the repo-local fallback — where the config roots are ~/alpha-engine-config and <repo_root>/../alpha-engine-config. Returns the ordered candidate list by default, or (with resolve=True) the first existing path, raising FileNotFoundError with every searched path named. Per-repo specifics are preserved as opt-in options with majority-default behavior: github_workspace adds research's $GITHUB_WORKSPACE/alpha-engine-config CI root, resolve_symlinks switches to executor's os.path.realpath+isfile semantics, exclude_suffixes=(".example",) is executor's never-resolve-the-template guard, and repo_local_fallback/extra_fallbacks cover the subdir-flattened (config/<file>) and multi-fallback local layouts. IO-agnostic (returns paths, not parsed config) so each consumer keeps its own parse/validate tail. Stdlib only.
quant — portfolio analytics engine (factor risk, VaR/CVaR, attribution, returns)
The shared institutional-analytics engine: pure, front-end- and data-source-agnostic functions that describe and measure a portfolio (performance, risk, attribution) with no advisory logic — it sits on the "analytics, not advice" side of the line. Lifted from robodashboard's analytics/ after the 2026-06-03 cross-repo leverage audit, so both the alpha-engine fleet and robodashboard consume one engine instead of parallel reimplementations. Import the submodule you need (the package keeps no eager imports, so the stdlib-only modules import without numpy):
quant.factor_risk— statistical factor risk modelΣ = B·F·Bᵀ + D, Option B (time-series factor-ETF estimator):estimate_factor_model(regress holdings on given factor return series),portfolio_risk(ex-ante vol + factor/idio split + per-factor variance contribution),tracking_error,benchmark_exposure, and a numpy-onlyledoit_wolf_cov(no sklearn). The estimator-agnostic consumption core (portfolio_risk/tracking_error) consumes anyFactorRiskModel(B, F, D). Needs numpy —pip install "nousergon-lib[quant]".quant.factor_risk_xs— sameΣ = B·F·Bᵀ + Dmodel, Option A (universe-wide cross-sectional Fama-MacBeth estimator): take exogenous per-ticker loadingsBand infer factor returnsf_tvia a cross-sectional OLS at each date →F/D(build_factor_risk_model,cross_sectional_factor_returns,estimate_factor_covariance,estimate_idiosyncratic_variance). Needs pandas + scikit-learn —pip install "nousergon-lib[quant-xs]"(kept separate so numpy-only consumers stay light).quant.risk_measures— parametric (Gaussian, Acklam inverse-normal, no scipy) + historical VaR & CVaR, as positive loss fractions at a horizon (stdlib).quant.riskstats—volatility,sharpe_ratio,sortino_ratio,downside_deviation,max_drawdown(stdlib). The fleet's only implementation of these statistics (config-I7597); variants are arguments (periods_per_year,denominator), never second implementations.quant.returns—xirr(money-weighted, Newton + bisection),time_weighted_return(GIPS),cumulative_return,annualize(stdlib).quant.seam_spread— one number per pipeline seam: output-population performance minus input-population performance, over the same window (alpha-engine-config#7214).seam_spread(...)returns aSeamSpreadcarrying the headline plus every per-date row it averaged;date_clustered_mean(...)is the estimator (equal weight within a date, equal weight across dates — "weeks-as-N");recompute_spread(published)reproduces the headline from the published record alone, importing nothing from the producer. Conventions are enforced, not documented: the output must be a subset of the input on the same date, a shared name must carry the same outcome on both sides of a selection seam (require_outcome_agreement=Falsefor a measurement seam such as planned-vs-filled or simulated-vs-realized), unresolved outcomes are counted and surfaced asunresolved_rate/unresolved_rate_gaprather than dropped silently, and a date present on only one side is excluded and named. Stdlib only — no extra required.quant.attribution— single-period Brinson-Fachler decomposition (brinson_fachler) + multi-period Cariño linking (link_periods) (stdlib).quant.stats— strategy/signal-quality evaluation metrics (lifted from the backtester'sanalysis/):dsr(Probabilistic + Deflated Sharpe, López de Prado),information_coefficient(Spearman rank IC),expectancy(hit-rate × win/loss decomposition),multiple_testing(Benjamini-Hochberg FDR),risk_matched_benchmark(EW-high-vol + beta-matched-SPY baselines + Information Ratio),regime_sortino(regime-stratified cross-sectional pick-alpha Sortino). Needs pandas + scipy —pip install "nousergon-lib[quant-stats]"(scipy is only the IC p-value; numpy fallback otherwise).
arena — the shared champion/challenger scoring engine
nousergon_lib.arena is the fleet's single implementation of
nous-ergon-ops/policies/champion-challenger-policy.md, so all four swappable slots — universe cut, selection producer, model (M), strategy (S) — run one set of rules instead of four drifting copies. Pure compute: stdlib only, no numpy, importable from a Lambda.
arena.arms— the append-only, immutable arm register. An arm is a recipe (features, hyperparameters, training-window rule and refit cadence) and its id encodes its own spec hash, so a changed recipe is necessarily a NEW arm and cannot inherit a record. A scheduled refit is recorded and changes nothing: the score series stays continuous across every refit. Lifecycle is an event fold, not a mutable row, soretired_dateis queryable without there being a field to overwrite.arena.ladder— the per-arm score ladder: 1, 2, 3, … N-week scores, every rung recomputed every cycle. The track record. There is deliberately no best-rung selector; picking the flattering horizon would turn a pre-registered statistic into a search over 52 of them.arena.window— longest-common-window pairing. Two arms are compared only over dates on which both produced, paired per date, and the intersection is reported alongside the metric. An empty intersection isunmeasurablewith a reason, never a tie.arena.confseq— an anytime-valid confidence sequence (Robbins normal-mixture boundary, Howard et al. 2021 §3.5) on the paired per-date difference. Valid at every stopping time, so a weekly look does not inflate the false-promotion rate the way 52 fixed-sample tests a year do. It subsumes minimum-evidence floors, which is whythin_evidence-style gates are removed rather than retuned.arena.ranking— Condorcet-style pairwise-wins ranking, which is how arms of very different ages are ranked without ever comparing incomparable windows: every pair is judged on its own overlap, and an arm's standing is its count of pairwise losses.arena.engine—run_cycle(): the pointer decision (free movement in both directions, no cooldown), the cap-with-grace retirement rule, hard serving preconditions that outrank any lead, a hardTrainingIntegrityErrorwhen any arm's fit is unsound, and thearena_cycleartifact.
from nousergon_lib.arena import ArenaConfig, ArmRegister, ArmSeries, run_cycle
gates — the phase-gate engine (clauses, readings, ladder)
nousergon_lib.gates is the fleet's single gate engine, lifted from crucible/crucible/gate.py on
its second adoption (data collector plan §4.1, alpha-engine-config-I10748). Three rules, carried
over verbatim in meaning:
- A gate reads; it never runs. Every clause is evaluated against artifacts already written, so a merge can never satisfy a gate.
- A clause is MET, UNMET or UNMEASURABLE, and UNMEASURABLE is never met. An absent artifact is UNMET with the missing key named; a read that failed is UNMEASURABLE, which is a fact about our reading and renders red rather than being folded into "unmet".
- The gate's job succeeds when the MEASUREMENT succeeds. The ladder is written and the process exits non-zero unless the gate is met, so nobody reads "not there yet" as "done".
gates.clause—Clause,unmeasurable(), andcontained()/contain_clause_exceptions(): every_clause_*function in a module is wrapped at import so a clause that raises becomes one UNMEASURABLE row instead of darkening the whole ladder. One unguarded client construction in one crucible clause took down every phase gate in the system on 2026-09-09; the containment lives in the engine so a clause author cannot forget it.gates.result—GateResult:met_ratioisNone, never0.0, when nothing was measured or when any clause is unmeasurable — a ratio computed over a partial read is an overclaim.gates.ladder—Phase,build_ladder(),ladder_payload()and the shippedphase_ladder.v1JSON Schema. Five states (MET/UNMET/UNMEASURED/UNMEASURABLE/OUT_OF_ORDER), each with a declaredobservability-policy§8.3 console rendering;UNMEASUREDrendersUNREPORTEDand counts against the transparency gap.gates.faults—fault_excused_run_ids(): only aninducedrecord, matched onrun_idalone, excuses anything. This is the one mechanism in a gate capable of turning a red clause green, so its refusals are the design.gates.store— a two-methodGateStoreProtocol, so the engine binds to no repo's store class, plus reads that keep absent and could not read apart.
No clause definitions live here. They stay in the repo that owns the thing being graded
(architecture.d/146 rule 1) — crucible grades crucible, nousergon-data's data_gate grades
the data collector. Needs the gates extra (jsonschema) for ladder validation.
from nousergon_lib.gates import Clause, GateResult, Phase, build_ladder, ladder_payload
gates.report / gates.tracker — the daily accountability report core
nousergon_lib.gates.report is the system-agnostic half of a daily report that reads artifacts it
does not own and pushes a pointer at an operator, lifted from crucible/crucible/morning.py on its
second adoption (alpha-engine-config-I10951). Imported by path, not re-exported from
nousergon_lib.gates, because Read and DocumentRead are different records for different jobs.
staleness()— the bolded first-line headline. An unparseable or absentgenerated_atis STALE; freshness is never assumed from a timestamp nobody could read.read_optional()→Read— present / absent / DENIED as three distinct facts.read_required()raises. A denial rendered as absence hides an IAM gap behind a normal-looking report — measured live, since S3 answers 403 for a missing key when the caller also lackss3:ListBucket, so absence is a listing, not a get.moved_since()→MovedResult— the previous-reading diff, withABSENT/VANISHEDsentinels and acannot_saythat is never rendered as "nothing moved".wire_budget()/assert_within_budget()— the transport-prefix arithmetic against whatkrepis.alertsactually prepends, pinned by a test against krepis' own formatter. Over budget RAISES; it never truncates, because a truncated report arrives reading complete.deliver()—silent=False,dedup_key=None,sns=False, an EXPLICIT destination, and anUndeliveredErroron the muted/dedup-suppressed publish krepis reports asany_ok=True.filter_withheld_clauses()— a withheld clause is REPLACED by a counting marker, never silently dropped.resolve_trigger()—GITHUB_EVENT_NAMEfirst, because a workflow's ownenv:block cannot forge a reserved prefix.history_row()/render_history_index()— the rolling index, last row per trading day, newest first, corrupt rows named in place.
nousergon_lib.gates.tracker is the rolling-issue adapter it posts through: Tracker over a
TrackerConfig(repo, token_var, app_ssm_prefix_var), find-or-create by title (two open issues with
that title is a loud error, never a pick), comment, and a body PATCH whose payload is a literal
{"body": ...}. It carries no close method — closing an issue is a human's authority, and the
absence of the method is the control. Credential: the named env var, else a short-lived installation
token minted through nousergon_lib.github_app narrowed to issues: write.
The ordering invariant belongs to the caller: post the tracker comment BEFORE rendering the
headline, because the headline's indispensable content is that comment's permalink. Both tracker
calls raise, so a failed post means the job's manifest is failed and no message is sent.
egress.routes — published LLM egress-proxy route contract
nousergon_lib.egress.routes publishes, as a versioned artifact with a JSON Schema beside it, which upstream hosts a multi-tenant LLM egress-proxy deployment serves and how each is authenticated — box_upstream_hosts(), laptop_upstream_hosts(), upstream_hosts(table), table(name), load_contract(), load_schema(). A request naming a host absent from the table is refused by the proxy with unknown upstream host, so anything deciding which model rows are servable has to know the table; publishing it here is what lets a consumer read it with no credential instead of checking out the private repo that configures the proxy (alpha-engine-config-I8337). The artifact carries upstream host, path prefix and auth mode only — no key-environment names, no ports, no host of ours — and tests/test_egress_routes_contract.py asserts that. Deployments are never unioned: box and laptop are different tables. Stdlib only.
testing.debug_swallow_guard — class guard for invisible exception swallows
find_debug_only_swallows(source_dir) AST-walks a directory of *.py files for except Exception
handlers whose entire body is a bare pass or a logger.debug(...) call — the shape that goes
unrecorded anywhere when a repo's root logger runs at INFO (alpha-engine-config-I10031,
crucible-executor-PR547). load_allowlist, check_against_allowlist and
check_allowlist_entries_self_contained diff the live sites against a repo-local
.debug-swallow-allowlist.yaml (same schema_version: 1 shape as .provider-linkage-allowlist.yaml)
so a consumer's own tests/test_no_debug_only_swallows.py is a few lines calling these four
functions rather than a copy of the AST walk (alpha-engine-config-I10226 — lifted on second
adoption per policy-shared-code once the class was measured in five sibling repos). Stdlib +
pyyaml only.
http_retry — bounded-backoff transient-API retry chokepoint
request_with_retry(url, *, params, session, transient_status, ...) returns the final requests.Response after retrying the transient class — 429 + 5xx responses (honoring Retry-After) and Timeout/ConnectionError network errors — with exponential backoff + full jitter; an exhausted network error raises HttpRetryError (api-key-scrubbed), while a persistent transient-status response is returned for the caller to interpret (so a 403, not in the transient set, is handed back for e.g. polygon's PolygonForbiddenError conversion). Also exposes the low-level backoff_delay(attempt, *, base, cap, retry_after) and scrub_api_keys(msg) (masks api_key=/apiKey= querystring values) for consumers with bespoke loops (the rate-limited polygon_client keeps its own loop + 403 + JSON parse and reuses just the delay math + scrubber). Consolidates the four mirrored alpha-engine-data retry sites (FRED fetch, polygon client, preflight reachability, FRED repair) into one policy so they stop drifting (L4499). Stdlib + requests only.
from nousergon_lib.quant.risk_measures import historical_cvar
from nousergon_lib.quant.factor_risk import estimate_factor_model, portfolio_risk
How it's used
All six Nous Ergon module repos depend on this lib:
| Module | Repo | What it imports from here |
|---|---|---|
| Data | alpha-engine-data |
logging, preflight, arcticdb, dates, trading_calendar, rag (ingestion), ec2_spot + ssm_log_capture + ssm_dispatcher (spot launchers) |
| Research | alpha-engine-research |
logging, decision_capture, cost, dates, rag (retrieval), agent_schemas (canonical LLM-output contracts) |
| Predictor | alpha-engine-predictor |
logging, preflight, arcticdb, dates, ec2_spot + ssm_log_capture + ssm_dispatcher (spot launcher) |
| Executor | alpha-engine |
logging, preflight, arcticdb, dates, trading_calendar |
| Backtester | alpha-engine-backtester |
logging, preflight, arcticdb, dates, agent_schemas (replay-harness Pydantic validation), ec2_spot + ssm_log_capture + ssm_dispatcher (spot launcher) |
| Dashboard | alpha-engine-dashboard |
logging, arcticdb, dates, hosts the SSM-target .venv that ssm_dispatcher invokes via python -m |
Development
git clone https://github.com/nousergon/nousergon-lib.git
cd nousergon-lib
pip install -e ".[dev,arcticdb,flow_doctor]"
pytest
Scope discipline
This repo is intentionally narrow. Code lands here when at least two consumers would otherwise maintain their own copy. New modules land as their own minor release with per-consumer adoption — no lockstep updates.
Code that does not belong here:
- Anything tunable (scoring weights, risk thresholds, sizing parameters) →
alpha-engine-config(private) - Agent prompts →
alpha-engine-config(private) - Module-specific business logic → that module's repo
License
AGPL-3.0-only — see LICENSE and NOTICE. Versions up to and
including 0.59.8 were released under the MIT License and remain available
under those terms; all subsequent versions (0.60.0+) are AGPL-3.0-only.
Commercial licenses are available — contact brian@nousergon.ai. External
contributions require DCO sign-off under the MIT inbound license (see
CONTRIBUTING.md).
Release files for nousergon-lib 0.124.153
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| nousergon_lib-0.124.153.tar.gz | 1.0 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| nousergon_lib-0.124.153-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.8 MB
Release files / nousergon_lib-0.124.153.tar.gz
| Download URL | nousergon_lib-0.124.153.tar.gz |
|---|---|
| Size | 1.0 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5960790bb010da639e5b2f03b85504eaaf90bc029b9f81a1da944eae114de12d
|
|
BLAKE2b-256 checksum How to use checksums |
6d07dc1435369b56cb79b6c6c613ecf8ab604bedfb089b977fb1fb860d2724e3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency logRelease files / nousergon_lib-0.124.153-py3-none-any.whl
| Download URL | nousergon_lib-0.124.153-py3-none-any.whl |
|---|---|
| Size | 718.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3b8abc9a90af760f6a8e309d503f6297b24f111e7d9317320a82386b42c5f8dc
|
|
BLAKE2b-256 checksum How to use checksums |
0ef183d971812b5f0e9a87b2c7d1efd16e2dce5fe8bd3b7a2913e42f8545808c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency log