promptwatt 0.1.0a2

Measured, rebound-aware lifecycle energy profiling and prompt optimization for local LLM inference.

PromptWatt

Measured lifecycle truth for prompt optimization.

PromptWatt profiles local LLM inference, detects energy rebound after prompt compression, and selects prompt variants only when they save measured energy after search cost, output expansion, quality, constraints, and expected reuse are accounted for.

The project does not claim that fewer input tokens automatically save energy. It records how every value was obtained: direct measurement, baseline-adjustment, estimation, proxy, or unavailable evidence.

Status: pre-release research alpha. The core API and schemas are tested; publish a tagged 1.0 only after cross-GPU and external-meter validation described in the validation plan is complete.

What is different

• Zeus and direct NVML backends, plus an external cumulative-meter adapter.
• Synchronized measurement windows and no silent zero samples.
• Gross energy as the primary physical observation; optional resident-idle baseline adjustment is clearly labeled and retains the signed result.
• Explicit Hugging Face prefill and KV-cached decode tracing. No one-token run is subtracted from a separate full generation.
• Whole-batch vLLM attribution with a warning when request isolation is absent.
• Constraint extraction with source spans; no invented fallback constraints.
• Real max_new_tokens budget enforcement, separate from prose instructions.
• Rebound classification from paired input, output, quality, and energy data.
• Optimization that measures the target model and amortizes the entire search.
• Stable SHA-256 row identities and atomic, preemption-safe result fragments.
• Paired randomization and cluster-bootstrap confidence intervals.

Install

pip install promptwatt
pip install "promptwatt[zeus,hf]"       # recommended NVIDIA + Transformers path
pip install "promptwatt[nvml,hf,stats]" # direct NVML and scientific extras

PromptWatt has no mandatory hardware, ML-framework, or measurement-backend dependency. On Python 3.10, it installs tomli as a lightweight compatibility dependency. Hardware and inference engines are optional extras and are imported lazily. PromptWatt never installs packages at runtime or import time.

Quick start

Create the monitor after loading and warming the model, so any baseline is calibrated in the actual resident state:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

from promptwatt.adapters import GenerationSettings, HuggingFaceAdapter
from promptwatt.measurement import EnergySession, ZeusBackend

model_id = "Qwen/Qwen2.5-0.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
if not torch.cuda.is_available():
    raise RuntimeError("This example requires an NVIDIA CUDA GPU.")

device = torch.device("cuda:0")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
).to(device)
model.eval()

# Warm the actual resident model state before constructing the energy backend.
warmup = tokenizer("Warm-up inference.", return_tensors="pt").to(device)
with torch.inference_mode():
    model.generate(**warmup, max_new_tokens=8, do_sample=False)
torch.cuda.synchronize(device)

adapter = HuggingFaceAdapter(model, tokenizer)
session = EnergySession(ZeusBackend([0]), synchronizer=adapter.synchronizer)
settings = GenerationSettings(max_new_tokens=96)

try:
    profiled = adapter.profile_phases(
        "Explain KV caching in exactly two sentences.", settings, session
    )
    for phase in profiled.trace.phases:
        print(phase.name, phase.reading.gross_j, phase.reading.evidence.value)
finally:
    session.close()

For an idle-adjusted sensitivity analysis, use direct power polling:

from promptwatt.measurement import NVMLBackend

backend = NVMLBackend(mode="power", poll_interval_s=0.1)
# First measurement calibrates a robust 10-second model-loaded idle baseline.

Gross joules remain available even when active_j is reported. “Active” here means resident-baseline-adjusted device energy, not perfectly attributable compute energy.

Rebound-aware optimization

from promptwatt.optimize import (
    HeuristicCandidateGenerator,
    OptimizationConfig,
    ReboundAwareOptimizer,
)

optimizer = ReboundAwareOptimizer(
    adapter,
    session,
    HeuristicCandidateGenerator(),
    config=OptimizationConfig(
        repeats=5,
        expected_deployments=10_000,
        quality_floor=0.85,
    ),
)
report = optimizer.optimize(
    "Please answer the question. Use JSON and no more than 50 tokens.", settings
)
print(report.modified, report.selected_method, report.total_optimization_energy_j)

The built-in lexical evaluator is labeled proxy. For claims about task quality, provide a task metric or human evaluation through QualityEvaluator.

CLI

promptwatt doctor
promptwatt rebound --original-input 500 --optimized-input 300 \
  --original-output 80 --optimized-output 190 \
  --original-energy 12 --optimized-energy 18
promptwatt lifecycle --original-energy 12 --optimized-energy 9 \
  --search-energy 600 --deployments 10000
promptwatt hf-profile Qwen/Qwen2.5-0.5B-Instruct \
  --prompt "Summarize this in three bullets" --phases --backend zeus

Repository map

src/promptwatt/
  measurement/   backend contracts, NVML, Zeus, external meters, integration
  tracing/       sequential lifecycle spans
  adapters/      callable, Hugging Face, and vLLM inference
  constraints/   extraction, restoration, and decoding ceilings
  analysis/      rebound, lifecycle, statistics
  optimize/      candidate generation and measured selection
  integrations/  LLMLingua-compatible compression
  benchmark/     paired factorial runs, batching, atomic fragments
  diagnostics/   optional research plots
  provenance.py  runtime, package, and CUDA visibility manifest

Read the measurement contract, architecture, and validation plan before publishing comparative energy claims.

Scope and non-claims

PromptWatt measures energy inside explicit local hardware windows. It does not infer datacenter cooling, networking, embodied carbon, or provider-side energy from API token counts. Carbon conversion is available only when the caller supplies a grid-intensity assumption. Baseline-adjusted GPU energy is a useful sensitivity metric, not exact causal attribution.

Contributing and citation

See CONTRIBUTING.md. If this supports published work, use the metadata in CITATION.cff. PromptWatt is MIT licensed.

PromptWatt is maintained by Abhay Phani Rao, an independent researcher (ORCID 0009-0003-9495-7697).