OpenJoule

OpenJoule is a plant-level inference and control engine for AI infrastructure.

OpenJoule watches one plant. It estimates the state from the telemetry it has. It forecasts what an action will do. It can impose that action. Then it measures what happened.

A serving system such as vLLM connects through a plant adapter. The adapter is separate from the core package. Telemetry comes in. Actions go out.

A plant is the system you are controlling. Serving, memory, workload, and power sit in one model.

GitHub · Issues · AID paper

Install

You need Python 3.10 or newer.

pip install openjoule

You can add the NVIDIA power helper if you want NVML writes.

pip install "openjoule[power]"

Quickstart

You pass in a few gauges and run one control step. You do not need a trace file.

from openjoule import Engine, GenericMetricsAdapter

plant = GenericMetricsAdapter(stale_after=None)
plant.ingest({"gpu_util": 0.4, "watts": 280, "kv": 10}, time=1.0)

engine = Engine()
record = engine.control_step(plant, now=1.0)
print(record.imposed, record.reason)
print(engine.status(plant, now=1.0))

If kv is present, OpenJoule imposes a power cap. If it only sees utilization and watts, it holds.

You can also choose an action from one observation. You do not need a plant attached for that.

from openjoule import Engine, Observation

engine = Engine()
obs = Observation(values={"gpu_util": 0.7, "watts": 320.0, "mean_qps": 12.0})
action = engine.loop(obs)
print(action)

The git repo includes a sample log at traces/telemetry.jsonl. That file is not part of the pip package. The sample has utilization and occupancy, and it has no kv, so the default step holds.

from openjoule import Engine, ReplayPlant

plant = ReplayPlant("traces/telemetry.jsonl")
record = Engine().control_step(plant)
print(record.imposed, record.reason)

engine.forecast rolls the logged action forward. engine.intervene rolls an imposed action. The scripts in experiments/ show the difference.

What it does

Step	What happens
Estimate	OpenJoule builds a plant state from the telemetry you have.
Forecast	OpenJoule predicts what a chosen action will do.
Impose	OpenJoule sends an action the plant can accept, such as a power cap.
Measure	OpenJoule records the outcome and the decision.

These pieces move together. Memory changes how fast the plant can serve. That changes the queue. The queue changes power and heat. The action changes service again.

Adapters

Piece	What it is
Plant	This is the interface. It has capabilities, read, write, and health.
ReplayPlant	This reads a JSONL, Prometheus, or OTLP file.
GenericMetricsAdapter	This reads live gauges. It does not import a serving runtime.
VLLMPlantAdapter	This reads live gauges using vLLM names. It does not import vLLM.
PowerCapWriter	This writes a power cap through NVML or DCGM. You install openjoule[power] for NVML.

The core package does not depend on vLLM or TensorRT-LLM.

Safety defaults

control_step is the path that writes to a plant. By default it will not write in these cases.

• The reading is stale.
• The estimate is ambiguous. Utilization with no kv is one example.
• The plant cannot take that action.
• The last write was too recent.
• The write fails.

Each decision can stay in memory. You can also append it to a JSONL audit file.

Project layout

openjoule/      The engine, the adapters, estimate, and control live here.
experiments/    These scripts ask which state a decision needs.
tests/          These are the tests.
traces/         These are sample readings. They are in the git repo, not the pip package.

Status

This is version 0.0.1, and it is alpha. The API can still change. You can use it for experiments, for replay, and for a careful loop on one plant.

Paper

OpenJoule follows AID, which stands for AI Infrastructure Dynamics. The paper asks which state you must see before a prediction under a new action is reliable.

AID: A Framework for AI Infrastructure Dynamics

Contributing

Issues and pull requests are welcome on GitHub.

git clone https://github.com/joule-lat/OpenJoule.git
cd OpenJoule
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
ruff check openjoule tests
pytest

Python in this repo has no comments and no docstrings. The Ruff line length is 100.

License

OpenJoule is released under the Apache 2.0 license. You can read it in LICENSE.

Links

• The source is at https://github.com/joule-lat/OpenJoule
• Issues are at https://github.com/joule-lat/OpenJoule/issues