ocelgen

Generate realistic multi-agent workflow traces on demand. Any domain, any pattern, any LLM. Validated against OCEL 2.0 and PM4Py.

pip install open-agent-traces

ocelgen generate --pattern sequential --runs 50 --noise 0.2 --seed 42

1,500+ events in under 2 seconds. No API key needed for structural traces.

run-0000: "My order arrived damaged, what are my options?"
├── run_started                                              08:00:00.007
├── agent_invoked          researcher    gpt-4o              08:00:00.052
│   ├── llm_request_sent   "Search for refund policy..."     08:00:00.067
│   ├── llm_response       "The refund policy states..."     08:00:00.749
│   ├── tool_called        web_search    → policy found      08:00:01.705
│   └── tool_called        file_reader   → order history     08:00:01.898
├── agent_invoked          analyst       gpt-4o              08:00:02.281
│   ├── llm_request_sent   "Analyze refund eligibility..."   08:00:02.334
│   ├── llm_response       "Customer is eligible for..."     08:00:06.747
│   └── tool_called        calculator    → refund amount     08:00:08.819
├── agent_invoked          summarizer    claude-3.5-sonnet   08:00:09.680
│   ├── llm_request_sent   "Draft resolution response..."    08:00:09.717
│   └── llm_response       "Dear customer, we apologize..."  08:00:10.363
└── run_completed                                            08:00:10.369
    cost: $0.038 | 3,950 input + 2,516 output tokens | 5 LLM calls | 3 tool calls

What it generates

Each trace includes LLM prompts and completions, tool call inputs and outputs, agent reasoning chains, inter-agent messages, calibrated token counts, realistic timestamps, and cost estimates — the same data you'd see in LangSmith, Arize, or Braintrust.

3 workflow patterns:

Sequential:    Research → Analyze → Summarize
Supervisor:    Supervisor → [Worker A, Worker B, Worker C] → Aggregate
Parallel:      Split → [Worker A ‖ Worker B ‖ Worker C] → Aggregate

10 deviation types with ground-truth labels for anomaly detection: skipped steps, wrong tools, swapped order, timeouts, missing handoffs, extra LLM calls, wrong routing, repeated activities, inserted activities, wrong resources.

10 built-in enterprise domains — or define your own in YAML:

Enrich with any LLM

Plug in any OpenAI-compatible endpoint to fill traces with realistic content:

# Cloud (OpenRouter — default)
export OPENAI_API_KEY="your-key"
ocelgen enrich output.jsonocel --domain customer-support-triage

# Local (llama.cpp, Ollama, vLLM — no API key needed)
ocelgen enrich output.jsonocel -d customer-support-triage \
  --model local-model --base-url http://localhost:8080/v1

# Full pipeline: generate + enrich + upload to Hugging Face
ocelgen pipeline --domain customer-support-triage --namespace your-hf-username

Enrichment chains context across agent steps, reflects deviations in the generated content, recalibrates token counts and timestamps, and expands seed queries via LLM for diversity across runs.

Validated, not just generated

Every trace is checked by 5 validation layers — tested across all 10 domains, all 3 patterns, and the live HF dataset:

from ocelgen.generation.engine import generate
from ocelgen.validation import (
    validate_referential_integrity,
    validate_workflow_conformance,
)

result = generate("sequential", num_runs=50, noise_rate=0.3, seed=42)
assert validate_referential_integrity(result.log) == []
assert validate_workflow_conformance(result.log, result.template) == []

Traces load directly in PM4Py — the reference OCEL 2.0 process mining library:

pip install open-agent-traces[conformance]

import pm4py
ocel = pm4py.read.read_ocel2_json("output.jsonocel")

Define your own domains

Create custom domains in YAML — they merge with the 10 built-ins:

domains:
  - name: "hr-onboarding"
    description: "HR onboarding: collect docs, run checks, provision access"
    pattern: "sequential"
    runs: 30
    noise: 0.15
    seed: 50001
    user_queries:
      - "New hire starting March 15 as Senior Engineer"
    agent_personas:
      researcher: "You are an HR coordinator collecting new hire documentation"
      analyst: "You are a compliance officer verifying background checks"
      summarizer: "You are an IT provisioner setting up accounts and access"
    tool_descriptions:
      web_search: "Search HR knowledge base for onboarding checklists"
      file_reader: "Read employee records and compliance documents"

ocelgen pipeline --domain hr-onboarding --config domains.yaml --namespace your-hf-username

Pre-built dataset

Don't want to generate? Load 17,000+ events directly from Hugging Face:

from datasets import load_dataset

ds = load_dataset("juliensimon/open-agent-traces", "incident-response")

for event in ds["train"]:
    if event["run_id"] == "run-0000":
        print(f"{event['event_type']:25s} | {event['agent_role']:12s} | {event['reasoning'][:60] if event['reasoning'] else ''}")

Who is this for?

• Agent observability teams — build and test monitoring dashboards with realistic trace data
• ML researchers — train anomaly detectors on labeled conformant vs deviant traces
• Process mining researchers — apply OCEL 2.0 conformance checking to multi-agent systems
• Agent framework developers — test LangGraph, CrewAI, AutoGen, Smolagents pipelines
• Evaluation teams — benchmark agent reasoning quality across domains and architectures

Examples

Documentation

• Quick Start — first dataset in 5 minutes
• User Guide — CLI reference, patterns, domains, custom YAML, validation, PM4Py
• Dataset on HF — 17,000+ events across 10 domains

Development

git clone https://github.com/juliensimon/ocel-generator.git && cd ocel-generator
uv sync --extra dev
uv run pre-commit install   # ruff + mypy + pytest on every commit
uv run pytest               # 265 tests, 98% coverage

License

MIT