AISimulate

AISimulate predicts LLM serving behavior and searches for deployment configurations offline, without bringing up a GPU serving cluster. The Python package includes the simulation engine, performance estimator and profiles, Replay, Sweeper, and deployment generation through the compatibility CLI and generator SDK.

Website · Repository · CLI guide

Install

Use Python 3.11–3.13:

python3 -m pip install aisimulate
aisimulate --help

The built-in engine runs without Dynamo. See the installation guide for platform requirements, source builds, nightly artifacts, and compatible Dynamo installations. Documentation on main may describe features newer than the latest published wheel. The optional Dynamo Planner integration requires additional dependencies described in that guide.

Predict one deployment

Save the following as prediction.yaml:

engine:
  mode: aggregated
  model: Qwen/Qwen3-32B-FP8
  hardware: h200_sxm
  backend: vllm
  workers:
    aggregated: {}

aisimulate predict --config prediction.yaml --output-dir ./prediction

The CLI prints a serving summary and writes the full report to prediction/prediction.json. Prediction uses the built-in engine stack by default; --stack dynamo selects a separately installed compatible Dynamo runner.

Search for a deployment

Save the following as recommendation.yaml:

traffic:
  source: {type: synthetic, input_tokens: 1024, output_tokens: 128}
  load: {type: concurrency, concurrency: 10}
  stop: {requests: 100}

engine:
  mode: aggregated
  model: Qwen/Qwen3-32B-FP8
  hardware: auto
  backend: {choices: [vllm, sglang]}
  workers:
    aggregated:
      parallelism: {preset: default}

optimization:
  target: throughput_per_gpu
  hardware: h200_sxm
  constraints:
    max_candidate_gpus: 8

optimizer:
  max_trials: 8

aisimulate recommend --config recommendation.yaml --output-dir ./recommendation

This quickstart evaluates suggestions for the explicit 1,024-input/128-output-token, 10-concurrent-request workload above, with an eight-trial budget.

Inspect recommendation/recommendation.json for search results and recommendation/recommendations/ for concrete prediction YAML files to replay. The CLI guide covers workload inputs, latency constraints, and detailed output. To create deployment manifests and launch scripts, use the bundled compatibility CLI or generator SDK; see the deployment generation guide.

Documentation and coverage

• Replay and Sweeper — embed replay and configuration search through Python APIs.
• Core API — engine, performance-model, and memory contracts.
• FPE Support Matrix — forward-pass estimator coverage.
• E2E Accuracy Overview and FPM Accuracy Overview — published accuracy evidence.
• Development guide — source setup and tests.
• Contributing and Code of Conduct — contribution requirements and community guidelines.

Estimator coverage is separate from serving-accuracy and deployment validation. Use the published evidence for your workload and verify deployment choices with real benchmarks.

Legacy CLI compatibility

The wheel provides the legacy aiconfigurator command through AISimulate 0.13.0. Removal is targeted for AISimulate 0.14.0, after every remaining workflow has a verified replacement in the unified CLI. Use aisimulate for new prediction and recommendation workflows. The migration guide explains replacements and remaining differences; the legacy CLI guide documents retained commands.

The canonical Python imports are aisimulate and aisimulate_core. AISimulate 0.13.0 removes the aiconfigurator and aiconfigurator_core import namespaces; see the Python source migration guide for replacement imports. The legacy executable remains available as described above.

When upgrading from standalone AIConfigurator, remove the old distributions first so that AISimulate owns the installed files:

python3 -m pip uninstall -y aiconfigurator aiconfigurator-core
python3 -m pip install --upgrade aisimulate