iwa-monitor 0.3.0

Behavioral integrity layer for AI systems during training and inference

signAI

Runtime behavioral integrity monitoring for PyTorch models.

signAI adds a small monitoring layer around training and inference so you can detect suspicious model behavior in real time. The production layer supports:

• Local artifact mode with no server
• Hosted signAI cloud mode
• Self-hosted server mode
• Air-gapped private deployment mode

Only compact behavioral vectors are scored. Model weights, raw inputs, gradients, and outputs do not leave the customer machine.

What is in this repo

• signai/core/: research and scoring core
• signai/client/: production SDK for attaching, calibrating, saving, loading, and scoring monitors
• signai_server/: standalone REST server for hosted and self-hosted scoring
• signai/api/server.py: existing demo server kept for experiments

Install

SDK only, local mode:

pip install signai

SDK plus server dependencies:

pip install "signai[server]"

SDK plus server plus Postgres support:

pip install "signai[server,postgres]"

For local development in this repo:

python -m venv .venv
.venv\Scripts\activate
pip install -e ".[server]"

Quick Start

Local mode

Calibrate once:

from signai import monitor

m = monitor.attach(model, num_classes=10, device="cuda")
m.calibrate(clean_loader, device="cuda", phase="inference", calib_batches=200)
m.save("./integrity.json")

Load and score:

from signai import monitor

m = monitor.load(model, artifact="./integrity.json", device="cuda")

for x, y in test_loader:
    result = m.score_inference(x, y)
    if result.flagged:
        route_to_fallback(x)

Remote mode

Start the server:

signai-server serve --host 0.0.0.0 --port 8000 --storage ./artifacts

Load a monitor against that endpoint:

from signai import monitor

m = monitor.attach(
    model,
    num_classes=10,
    endpoint="http://localhost:8000",
    api_key="",
    monitor_id="demo-model",
    device="cuda",
)
m.calibrate(clean_loader, device="cuda", phase="inference", calib_batches=200)
m.save("./integrity.json")  # also uploads artifact in remote mode

Then use the same scoring API:

result = m.score_inference(x, y)
result = m.score_training(logits, loss)

Deployment Modes

1. Local artifact

m = monitor.load(model, artifact="./integrity.json")

2. signAI cloud

m = monitor.load(
    model,
    endpoint="https://api.signai.dev",
    api_key="sk-...",
    monitor_id="my-model",
)

3. Self-hosted

m = monitor.load(
    model,
    endpoint="http://signai.internal:8000",
    api_key="...",
    monitor_id="my-model",
)

4. Air-gapped

m = monitor.load(
    model,
    endpoint="http://10.0.1.5:8000",
    api_key="...",
    monitor_id="my-model",
)

Training Loop Integration

from signai import monitor

m = monitor.load(model, artifact="./integrity.json", device="cuda")

for x, y in train_loader:
    optimizer.zero_grad()
    logits = model(x)
    loss = criterion(logits, y)
    loss.backward()
    optimizer.step()

    result = m.score_training(logits, loss)
    if result.flagged:
        quarantine_update(result)

Inference Loop Integration

from signai import monitor

m = monitor.load(model, artifact="./integrity.json", device="cuda")

for x, y in test_loader:
    result = m.score_inference(x, y)
    if result.flagged:
        route_to_fallback(x)

Run The Server

CLI

signai-server serve \
  --host 0.0.0.0 \
  --port 8000 \
  --storage ./artifacts \
  --store-backend file \
  --edition community

Shortcut through the main CLI:

signai serve --host 0.0.0.0 --port 8000 --storage ./artifacts

Docker

docker build -t signai .
docker run -p 8000:8000 -v %cd%\artifacts:/data signai

docker-compose

docker compose up --build

API Summary

Core server endpoints:

• GET /health
• POST /v1/artifacts
• GET /v1/artifacts/{monitor_id}
• DELETE /v1/artifacts/{monitor_id}
• POST /v1/score/inference
• POST /v1/score/training
• POST /v1/score/batch
• POST /v1/calibrate/start
• POST /v1/calibrate/push
• POST /v1/calibrate/commit

Pro and enterprise endpoints:

• GET /v1/history/{monitor_id}
• POST /v1/notify/configure
• GET /v1/audit/export
• GET /v1/status
• GET /v1/monitors enterprise only

Editions

• community: SDK, local mode, file-backed server
• pro: history, notifier, audit export, status
• enterprise: Postgres and multi-node oriented deployment

Edition is controlled with:

SIGNAI_EDITION=community|pro|enterprise

Privacy Model

signAI is built so that privacy is enforced structurally:

• feature extraction runs on the customer machine
• remote scoring receives only s and z float vectors
• the server discards raw vectors after scoring
• only {ts, score, flagged, phase} is stored in history

Documentation

• User manual: USER_MANUAL.md
• Deployment guide: DEPLOY.md
• Changelog: CHANGELOG.md
• Release checklist: RELEASE_CHECKLIST.md
• Product strategy: PRODUCT_STRATEGY.md
• Go-to-market plan: GO_TO_MARKET.md
• Commercial plan: COMMERCIAL_PLAN.md

Productization

The product vision is larger than a research repo:

• every ML system should be able to add an integrity monitor
• developers should be able to discover it globally
• teams should be able to run it locally, in managed cloud, self-hosted, or air-gapped
• community adoption should come from local-first usability
• paid adoption should come from hosted operations, team workflows, and enterprise delivery

Development Notes

Run the targeted test suite:

python -m pytest tests\test_imports.py tests\test_local_backend.py tests\test_monitor.py tests\test_server.py

License

AGPL-3.0 for open-source use. Commercial licensing can be layered on top for closed-source distribution.