Skip to main content

Contracts and reference tooling for reproducible agentic environments.

Project description

Reproducible Agentic Environments System

Reproducible Agentic Environments System (RAES) is a system for describing, realizing, controlling, evaluating, and reproducing agentic environments. An agentic environment is a declared and realized setting in which participants receive observations, take actions, interact with resources or other participants, and are evaluated under stated controls. Participants can include software agents, policies, scripts, and human-control proxies.

RAES supports a bounded reproduction attempt by connecting authored intent, deterministic or governed variation, realization inputs, participant behavior, observations, apparatus identity, provenance, evidence, replay boundaries, and conformance results. It does not guarantee deterministic runtime behavior, equal outcomes, exact replay, scientific validity, or reproducibility.

Cyber, AI security, AI safety, testing, research, and evaluation are non-exhaustive application areas. The general model can support additional domains through domain-specific examples, controlled vocabularies, semantic profiles, reusable assets, backend profiles, and evidence requirements.

The current repository materializes RAES through its Scenario Description Language (SDL), Python reference implementation, published contracts, examples, and assurance material. RAES names the overall system; RAES SDL is the authored scenario language within it.

The repository separates authored scenario meaning from processors, backends, participant implementations, runtime state, and archived evidence. In the current implementation, an SDL document can be parsed, validated, instantiated, compiled into runtime models, and checked against published backend contracts without binding the authored scenario to one cloud, range implementation, or execution harness.

This is an academic and engineering project. The repository is intended to be read, tested, and used as reference implementation code, not treated as a managed service.

The repository is not a managed environment service and does not include a production backend. Backend contracts, stubs, conformance checks, and examples are present; real deployment backends remain separate implementations. Cyber is the strongest current example and lineage base, not the boundary of the core.

A worked example of RAES SDL driving a concrete range is APTL (Advanced Purple Team Lab), a separate project that specifies its scenarios as RAES SDL documents and realizes the selected topology on a Docker Compose backend.

Contents

Agentic Environments And RAES SDL

RAES SDL records authored scenario and experiment intent. An SDL file can describe topology, hosts, services, identities, content, relationships, agents, objectives, workflows, variables, and evaluation material without directly describing a specific backend's infrastructure primitives. Processors, backends, participant implementations, and runtime choices turn that authored scenario into a realized environment; the realization is not identical to the SDL document.

name: hospital-ransomware-surgery-day
description: Surgery-day ransomware exercise for a regional hospital.

variables:
  surgery_day_speed:
    type: number
    default: 1.0

nodes:
  internet-edge:
    type: Switch
    description: Public ingress for email, VPN, and external access

  mail-gateway:
    type: VM
    os: linux
    source: secure-mail-gateway
    resources: {ram: 2 gib, cpu: 1}
    services:
      - {port: 25, name: smtp-inbound}
    roles: {mail-admin: postfix}

Complete examples live in examples/scenarios/. Reusable non-normative templates and patterns are indexed by examples/library/catalog.yaml.

Getting Started

Prerequisites:

  • Python 3.11 or newer
  • uv
  • nox for the repository verification graph, or uvx nox without a separate install

Set up the Python reference implementation:

git clone https://github.com/RAESystem/rae.git
cd rae/implementations/python
uv sync --all-extras
uv run raes --help

Using the Python Reference Implementation

Parse and validate a scenario from Python:

from pathlib import Path

from raes import parse_sdl_file

scenario = parse_sdl_file(
    Path("../../examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml")
)

for advisory in scenario.advisories:
    print(advisory)

Run the CLI from implementations/python:

uv run raes sdl resolve ../../examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml
uv run raes sdl verify-imports ../../examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml
uv run raes sdl publish ../../examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml
uv run raes processor --help
uv run raes conformance --help
uv run raes-mcp

Repository Layout

  • specs/ - normative prose and formal specification material
  • contracts/ - published schemas, fixtures, manifests, and profiles
  • implementations/ - reference implementations and their local tooling
  • examples/ - worked SDL scenario examples plus reusable authoring templates and patterns
  • docs/ - explanatory documentation, API docs, and architecture decisions
  • research/ - supporting literature and reference ecosystem material
  • tools/ - repository maintenance, policy, and publication tooling

Lineage

For a dimension-by-dimension comparison against these systems — what RAES expresses that they do not, and where they still lead RAES — see Related-Work Comparison.

Documentation

The documentation source is under docs/. Important entry points:

Verification

nox is the canonical verification graph. From the repository root:

uvx nox -s verify
uvx nox -s verify-changed
uvx nox -s tests
uvx nox -l

The full verify session is unconditional and runs the project checks expected for pull requests, including repository policy, governed evidence and generated artifacts, parallel worker-safe test coverage, serial integration tests, and docs. Use verify-changed while iterating: it compares the branch and working tree with the upstream ref, skips evidence and regression stages only for allowlisted prose or research-only changes, and fails closed to the full local gate for unknown, deleted, renamed, executable, contract, or configuration changes. The pre-push hook uses the same change-aware plan.

Whole-scenario finite-domain satisfiability can be inspected without applying or provisioning a scenario:

uv run --project implementations/python raes processor satisfiability \
  path/to/scenario.sdl.yaml \
  --profile aces-finite-domain-satisfiability-v1

The command emits the published replayable evidence envelope. Exit 0 is a completed satisfiable or unsatisfiable analysis, 2 is an explicit unsupported fragment result, and 1 is an input or operational failure. See ADR-086 for the bounded target coverage and nonclaims.

Contributing

Contributions are welcome where they improve the language, reference implementation, contracts, tests, examples, or documentation. Start with CONTRIBUTING.md.

Language and contract changes should be discussed before implementation because small SDL changes can affect validation, generated schemas, backend conformance, and existing scenario examples.

Versioning

The Python package version lives in implementations/python/packages/raes/_version.py and is bumped by release-please from the Conventional Commit history on main, which also generates CHANGELOG.md. Do not hand-edit the version or CHANGELOG.md. See docs/explain/releasing.md.

Published JSON Schemas use versioned contract identifiers such as sdl-authoring-input-v1, but the suffix is not the same as a stability promise. The authoritative schema publication index assembles independent per-contract records, each carrying its schema's draft or stable stability class and canonical content hash. Current checked-in schemas are draft until a maintainer explicitly promotes them; stable breaking changes must mint a new schema version as described in ADR-061.

Maintainers

Citation

If you use RAES in academic work, cite the system:

@software{raes,
  author       = {Edwards, Brad},
  title        = {RAES: Reproducible Agentic Environments System},
  year         = {2026},
  license      = {MIT},
  url          = {https://github.com/RAESystem/rae}
}

License

Released under the MIT License. See LICENSE. Third-party attribution and license notices are recorded in THIRD_PARTY_NOTICES.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

raes-1.1.0.tar.gz (2.2 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

raes-1.1.0-py3-none-any.whl (2.0 MB view details)

Uploaded Python 3

File details

Details for the file raes-1.1.0.tar.gz.

File metadata

  • Download URL: raes-1.1.0.tar.gz
  • Upload date:
  • Size: 2.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for raes-1.1.0.tar.gz
Algorithm Hash digest
SHA256 fb84bebd677e907d70171c8ef098bac59c856ac433b72a003af22c1c65175b93
MD5 60af53a6ee248134f75f7f0f1b3a0211
BLAKE2b-256 656719c5b64b5cec853e07bd339694a015ba3ec2b115752e82c6620f6918497c

See more details on using hashes here.

Provenance

The following attestation bundles were made for raes-1.1.0.tar.gz:

Publisher: release-please.yml on RAESystem/rae

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file raes-1.1.0-py3-none-any.whl.

File metadata

  • Download URL: raes-1.1.0-py3-none-any.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for raes-1.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 8cc01e59a5b58a92a649cc9b6b47d51be80adb193de5c9829d5e52cdd25939fb
MD5 1a132a17538fd30bbc31359ce0f5058e
BLAKE2b-256 11d373924a63465ed939aaf58cb720893d914d9712fa77bf0797a017f3c0289c

See more details on using hashes here.

Provenance

The following attestation bundles were made for raes-1.1.0-py3-none-any.whl:

Publisher: release-please.yml on RAESystem/rae

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page