iwa-monitor 0.3.4

Behavioral integrity layer for AI systems during training and inference

signAI

Runtime behavioral integrity monitoring for PyTorch models.

signAI detects adversarial attacks, poisoned updates, and anomalous inputs at training and inference time — without sending model weights, raw inputs, or gradients anywhere. Only compact behavioral vectors are scored.

Install

pip install iwa-monitor

With PyTorch support:

pip install "iwa-monitor[torch]"

Start the Daemon

The daemon handles calibration and stores artifacts. Pull and run it with Docker:

docker pull ghcr.io/nextkore/signai-daemon:latest
docker run -d -p 7731:7731 -v ~/.signai:/data ghcr.io/nextkore/signai-daemon:latest

The SDK auto-discovers the daemon on localhost:7731 — no configuration needed.

How It Works

The SDK runs entirely on the customer machine. It uses PyTorch hooks to extract behavioral signals from the model, then sends compact float vectors to the signAI daemon for calibration and scoring. The daemon never sees model weights, raw inputs, or gradients.

Your machine                          signAI daemon (localhost:7731)
─────────────────────────────         ──────────────────────────────
model forward/backward pass
      ↓
feature extraction (hooks)
      ↓
s, z float vectors           →        fit detector / score
                             ←        score, flagged, tau

Quick Start

Step 1 — Start the daemon

docker run -p 7731:7731 signai/daemon

The SDK auto-discovers the daemon on localhost:7731. No configuration needed.

Step 2 — Calibrate

import signai

m = signai.attach(model, num_classes=10, monitor_id="my-model", device="cuda")
m.calibrate(clean_loader, device="cuda", phase="inference", calib_batches=200)
m.save("./integrity.json")

Step 3 — Score at inference time

import signai

m = signai.load(model, artifact="./integrity.json", device="cuda")

for x, y in test_loader:
    result = m.score_inference(x, y)
    if result.flagged:
        route_to_fallback(x)

Step 4 — Score during training

m = signai.load(model, artifact="./integrity.json", device="cuda")

for x, y in train_loader:
    optimizer.zero_grad()
    logits = model(x)
    loss = criterion(logits, y)
    loss.backward()
    optimizer.step()

    result = m.score_training(logits, loss)
    if result.flagged:
        quarantine_update(result)

Deployment Modes

Local artifact (no server needed for scoring)

Calibrate once with the daemon, save the artifact, then score locally without the daemon running:

m = signai.load(model, artifact="./integrity.json", device="cuda")
result = m.score_inference(x, y)

Self-hosted daemon

m = signai.attach(
    model,
    num_classes=10,
    endpoint="http://signai.internal:7731",
    api_key="your-key",
    monitor_id="prod-model",
    device="cuda",
)

Air-gapped

Same as self-hosted — point endpoint at your internal daemon address.

Supported Models

Works with any PyTorch nn.Module:

• Standard CNNs, MLPs, RNNs
• HuggingFace Transformers (input_ids/attention_mask dict inputs, .logits outputs)
• Vision Transformers (ViT)
• Custom architectures — hooks are auto-selected from leaf modules

Calibration Phases

Phase	Detects	When to calibrate
inference	adversarial inputs, distribution shift	on clean test/validation data
training	poisoned updates, gradient manipulation	during a clean warmup training run

SDK Reference

import signai

# Attach without loading an artifact (for calibration)
m = signai.attach(model, num_classes=10, monitor_id="...", device="cuda")

# Load a saved artifact (for scoring)
m = signai.load(model, artifact="./integrity.json", device="cuda")

# Calibrate (requires daemon)
m.calibrate(loader, device="cuda", phase="inference", calib_batches=200)
m.calibrate(loader, device="cuda", phase="training", optimizer=opt, criterion=loss_fn, warmup_steps=200)

# Save artifact locally
m.save("./integrity.json")

# Score
result = m.score_inference(x, y)   # returns MonitorResult
result = m.score_training(logits, loss)

# MonitorResult fields
result.score     # float — Mahalanobis distance
result.flagged   # bool — True if score > tau
result.tau       # float — calibration threshold
result.phase     # "inference" or "training"
result.error     # str or None

CLI

# Check daemon status and usage
signai status

# Apply a license key
signai license <key>

# Show version
signai version

Daemon API

The daemon exposes a REST API on port 7731:

• GET /health
• POST /v1/calibrate/start
• POST /v1/calibrate/push
• POST /v1/calibrate/commit
• POST /v1/score/inference
• POST /v1/score/training
• POST /v1/score/batch
• GET /v1/artifacts/{monitor_id}
• GET /v1/artifacts/{monitor_id}/download
• DELETE /v1/artifacts/{monitor_id}
• GET /v1/usage
• POST /v1/license

Pro/Enterprise endpoints:

• GET /v1/history/{monitor_id}
• POST /v1/notify/configure
• GET /v1/audit/export
• GET /v1/status
• GET /v1/monitors — enterprise only

Editions

Edition	Included
community	SDK, daemon, local artifact scoring, 10 calibration runs/month
pro	+ history, notifications, audit export, status dashboard
enterprise	+ Postgres, multi-node, unlimited runs

License keys are applied via the daemon:

signai license <your-key>

Privacy

Feature extraction runs on the customer machine. The daemon receives only s and z float vectors — no weights, no raw inputs, no gradients. Raw vectors are discarded after scoring. History stores only {ts, score, flagged, phase}.

License

MIT