omnisim

Official Python SDK for OMNISIM Synthetic Users — register your staging API (OpenAPI) and run persona-driven tests against real HTTP endpoints.

Install

pip install omnisim

Web / hybrid runs

Web and hybrid runs auto-prepare agent personas before starting (no separate scrape step). After a successful run, a Swarm-style UX research PDF is saved to ./reports/ and opened in your browser. Use --no-report and --no-open-report to opt out.

omnisim-run run --agents 3 --scenarios explore

Quick start

Set required environment variables:

export OMNISIM_API_KEY="omni_..."      # Required
export OMNISIM_BASE_URL="..."          # Required (e.g., http://localhost:8000)

from omnisim import Omnisim

client = Omnisim.from_env()

product = client.products.register(
    name="Acme API",
    base_url="https://staging.api.acme.dev",
    openapi=open_api_dict,
    auth={"type": "bearer", "token": staging_token},
)

run = client.synthetic_users.run(
    product_id=product["id"],
    personas=50,
    scenarios=["signup"],
    mode="behavioral_only",
)

result = run.wait_for_complete()
print(result["friction_points"], result.get("recommendations"))

Two layers: short map + detailed code index

1. app_context (short) — You write a brief map: routes, auth, how to navigate.
2. Code index (detailed) — SDK scans the repo, uploads chunks + graph edges; at run time OMNISIM retrieves the relevant pieces (embeddings when HF_API_TOKEN is set on the API host).

# After register-web / register
omnisim-run index push --product-id prod_xxx --repo .
omnisim-run run --scenarios onboarding

Agents get both: Mapper-discovered app knowledge and code summaries — fewer blind read_screen loops, lower token burn.

How agents know your app (Mapper + Code Index)

No manual app description required. When you first run 5+ agents against a web/hybrid product, OMNISIM automatically:

1. Runs a Mapper agent (one time only) to explore auth, routes, and APIs
2. Saves discoveries to a structured map (auth works via magic link, /billing route exists, etc.)
3. Injects the map into all subsequent test agents — they start informed

from omnisim import Omnisim

client = Omnisim.from_env()
product = client.products.register(
    name="Acme",
    base_url="http://127.0.0.1:3000",
    app_url="http://127.0.0.1:3000",
    interaction_mode="web",  # or "hybrid"
)

# First run with >= 5 agents triggers auto-mapping
run = client.synthetic_users.run(product_id=product.id, agents=40, scenarios=["onboarding"])
# Mapper runs (~1-2 min), then 40 agents test with the saved map

# Second run: no Mapper delay, all 40 agents start with the map
run2 = client.synthetic_users.run(product_id=product.id, agents=40, scenarios=["billing"])

Pre-mapping (optional — avoids the ~1-2 min delay on first run):

omnisim-run map --product-id prod_xxx

Re-map after deploy (if routes or auth changed):

omnisim-run map --product-id prod_xxx --force

Check map status:

omnisim-run map --product-id prod_xxx --status

Layer 1: Code Index (optional, CI-powered)

For deeper knowledge, push your repo structure from CI:

omnisim-run index push --product-id prod_xxx --repo .

This adds file-level context (Next.js routes, component locations) that the Mapper can reference but not replicate from browser exploration alone.

Resources

• client.products — register / list / get / delete products (api, web, hybrid)
• client.synthetic_users — behavioral runs with poll backoff
• client.usage — GET /api/v1/usage (daily limits, tier stats)
• client.webhooks — run.complete callbacks (Pro / Pro+)
• verify_webhook() — HMAC verification for webhook payloads
• capture_browser_session() — capture authenticated Playwright session (manual login)
• client.products.ensure_browser_auth() — auto-attach session via staging bypass URL
• client.products.attach_browser_session() — attach session to web/hybrid product

API key tiers

Tier	Agent LLM	Browser read_screen
Free	Groq	ARIA accessibility tree
Pro / Pro+	Moonshot Kimi K2.5	Enhanced DOM (boxes + key styles)

Your API key tier is returned from client.usage.get()["tier"]. The hosted API must have MOONSHOT_API_KEY configured for paid-tier agent runs.

Full docs: docs/synthetic-users-sdk.md

Browser authentication (web/hybrid products)

Low-friction: staging bypass (CI-friendly)

Add a test-only endpoint on your staging app that sets session cookies and redirects:

GET /api/test/session?token=${STAGING_AUTH_SECRET}
→ Set-Cookie session=...; Redirect /dashboard

Then set the full URL (with token) in CI:

export STAGING_AUTH_BYPASS_URL="https://staging.app/api/test/session?token=..."
omnisim-run run --product-id prod_xxx --agents 3
# or explicitly:
omnisim-run ensure-browser-auth --product-id prod_xxx

omnisim-run run calls ensure_browser_auth automatically for web/hybrid when STAGING_AUTH_BYPASS_URL is set. Public apps with no bypass URL are unchanged (--no-ensure-browser-auth to skip).

client.products.ensure_browser_auth(product_id, strategy="auto")
client.icp.ensure_ready(product_id, agent_count=3, product=product)
run = client.synthetic_users.run(product_id=product_id, personas=3, scenarios=["explore"])

Optional product hints in user_flows._config.browser_auth:

{ "strategy": "dev_bypass", "bypass_url": "https://staging.app/api/test/session", "wait_for_url": "/dashboard" }

Manual capture (OAuth / form login)

For staging apps behind login where bypass is not available, capture once and agents reuse the session:

# Install with browser support
pip install omnisim[browser]
playwright install chromium

# Capture session (opens browser, you login, press Enter)
omnisim-run capture-session --app-url http://127.0.0.1:3000 --product-id prod_xxx

Or in Python:

from omnisim import Omnisim, capture_browser_session

client = Omnisim.from_env()

# Capture after manual login
state = capture_browser_session("http://127.0.0.1:3000")
client.products.attach_browser_session(product_id, state)

# All subsequent runs start authenticated
run = client.synthetic_users.run(product_id=product_id, personas=3, scenarios=["explore"])

Internal QA: Agent Evals (optional)

For testing the simulation quality itself — not your product — enable post-run agent evaluations:

# Install DeepEval first
pip install -r backend/requirements-eval.txt

# Run with evals (internal testing only, adds ~2-5s per agent)
omnisim-run run --product-id prod_xxx --agents 5 --evals

# Machine-readable eval results
omnisim-run run --product-id prod_xxx --agents 5 --evals --json | jq '.agent_evals'

Evals score both synthetic agent quality (task completion, persona consistency) and finding trustworthiness (narrative faithfulness to traces). Results attach to result_json.agent_evals and appear in the CLI report.

Also works with test-product:

omnisim-run test-product --product-id prod_xxx --scenarios explore --evals

See docs/agent-evals.md for full documentation including prerequisites, environment setup, and troubleshooting.

v0.2.0 highlights

• Omnisim.from_env() and HTTPS base URL validation
• Typed errors (OmnisimAuthError, OmnisimRateLimitError, …)
• Automatic retries on 429 / 502 / 503 / 504
• HMAC webhook signatures + verify_webhook()
• RunMode enum and exponential backoff in wait_for_complete()

Development

pip install -e "packages/sdk-python[dev]"
pytest packages/sdk-python/tests/

TypeScript

Optional mirror: @omnisim/sdk (npm). Python is canonical.