gnosys

Python client for Gnosys Labs — the autonomous experimentation platform that proposes ML pipelines, runs them, statistically validates the results against selection bias, and only promotes what survives.

Install

pip install gnosyslabs

Installed as gnosyslabs, imported as gnosys (like scikit-learn → sklearn). The bare PyPI name gnosys is an unrelated project.

Quickstart

from gnosys import GnosysClient

# API key from the dashboard at https://gnosyslabs.com/dashboard/api-keys.
# Or set $GNOSYS_API_KEY in your environment.
client = GnosysClient(api_key="gn_live_...")

# Kick off a hyperparameter sweep on a tabular classification problem.
run = client.runs.create(
    domain="tabular",
    strategist={
        "kind": "hp_sweep",
        "key": "C",
        "values": [0.001, 0.01, 0.1, 1.0, 10.0],
    },
    spec_template={
        "spec_id": "_t",
        "name": "LR sweep",
        "hypothesis": "regularisation strength",
        "task": "classification",
        "dataset_id": "synthetic_binary",
        "model_family": "logistic",
        "hyperparameters": {"C": 1.0},
    },
    max_iterations=4,
    no_progress_window=2,
)

# Block until the run finishes (or hit timeout).
run = client.runs.wait(run.run_id, timeout=600)
print(f"final tier: {run.status}, error: {run.error}")

# Pull the validation findings — same data shape `stigmera diagnose
# findings` shows in the internal CLI.
for finding in client.findings.list(run_id=run.run_id, severity="blocker"):
    print(f"[{finding.severity}] {finding.validator}: {finding.detail}")

What the platform does

You hand Gnosys a spec template (dataset + model family + hyperparameters) and it runs a closed propose-execute-validate-promote loop:

1. Propose — a strategist (rule-based HP sweep, or LLM-driven) emits the next batch of pipeline variants.
2. Execute — the platform fits and scores each one.
3. Validate — the bidirectional validation layer catches generators that fool their own evaluator: multi-calibrated ensembles, distribution-shift decomposition, four honest-eval verifiers (shuffled-label, randomized-feature, secondary-holdout, permutation-FWER) plus an opt-in pre-execute LLM critic.
4. Promote — surviving pipelines get tier-classified (library / candidate / paper).

Round-by-round validation findings flow back to the strategist as structured prose so the next round's proposals adapt.

API surface

The client wraps the public REST API at https://gnosyslabs.com/v1/*. Every method is available on both GnosysClient (sync) and AsyncGnosysClient.

Runs

run = client.runs.create(domain=..., strategist=..., ...)
run = client.runs.get(run_id)
run = client.runs.wait(run_id, timeout=600, poll_interval=2.0)
runs = client.runs.list(status="completed", limit=50)
iterations = client.runs.iterations(run_id)

Findings

findings = client.findings.list(
    run_id="...",                          # or pipeline_run_id
    validator="honest_eval.shuffled_label",
    severity="blocker",                    # info|low|medium|high|blocker
    spec_id="...",
    limit=100,
)

Correlations (cross-validator)

matches = client.findings.correlations(
    validators=["llm_critic", "honest_eval.shuffled_label"],
    severity="blocker",
    mode="and",   # specs flagged by ALL of them
)

API keys

keys = client.api_keys.list()
client.api_keys.revoke(key_id)

Errors

from gnosys import (
    GnosysError,            # base class
    AuthenticationError,    # 401 — missing / revoked key
    ForbiddenError,         # 403 — tenant suspended
    NotFoundError,          # 404 — wrong run id, or another tenant's
    RateLimitError,         # 429 — see error.retry_after
    ValidationError,        # 400 — bad request payload
    ServerError,            # 5xx — caught after retry exhaustion
)

try:
    run = client.runs.create(domain="tabular", ...)
except RateLimitError as exc:
    time.sleep(exc.retry_after or 60)

Async

from gnosys import AsyncGnosysClient

async def main():
    async with AsyncGnosysClient(api_key="gn_live_...") as client:
        run = await client.runs.create(...)
        run = await client.runs.wait(run.run_id)
        async for finding in client.findings.iter(run_id=run.run_id):
            ...

Configuration

Documentation

Full docs at docs.gnosyslabs.com, including quickstart guides for tabular classification, HP sweeps, LLM-driven discovery, and the validation-layer concepts (the deception test, the four honest-eval verifiers, and the explanations registry).

License

Proprietary. See LICENSE. Free to use against gnosyslabs.com; redistribution restricted.