Skip to main content

ShiftGuard

Stale does not mean unsafe. A deterministic simulator and a set of commit-time guards for tool-using LLM agents whose view of infrastructure state can go stale between read and commit.

CI arXiv Dataset

Paper: Stale Does Not Mean Unsafe: Guard Precision for Tool-Using LLM Agents under Infrastructure State Races. Zihao Zheng, Jiayu Long, Baichuan Li, Junyi Yao. arXiv:2609.29522.

Why

When an agent restarts a job, promotes a snapshot, or changes an ACL, the state it read may have changed by the time it commits. Not every change makes the commit unsafe. ShiftGuard separates three kinds of race:

  • invalidating: the change breaks a declared safety predicate, so the commit must be blocked.
  • predicate-preserving: state changed, but the predicate still holds, so blocking is a false alarm.
  • irrelevant: the change does not touch the action.

It then measures how precisely each runtime guard tells these apart. A coarse guard (for example a global epoch check) is safe but blocks a lot of benign work. A semantic commit predicate can be just as safe at a fraction of the cost.

What's in here

Path Contents
src/shiftguard/simulator.py 16 infrastructure tasks in 4 domains (scheduler, data pipeline, object store, access control), visible vs. authoritative state, 5 non-atomic fault injectors
src/shiftguard/controllers.py Baselines and guards: always_execute, fast/strong verification, escalate, TTL / confidence / contract / agreement gates, evidence_router, global_epoch_guard, read_set_version_guard, commit_predicate_guard plus ablations
src/shiftguard/prompts.py, ollama_client.py Proposal prompts and a dependency-free Ollama client for collecting new trajectories
scripts/ Collection, counterfactual replay, analysis, learned router, cost sensitivity, figures
tests/ 17 deterministic simulator and guard tests
examples/sample_trajectories.jsonl 72 frozen proposals from the main study, enough to run the full pipeline offline

Correctness is scored deterministically. No LLM judge is used anywhere.

Quick start

The core package and the replay pipeline use only the Python standard library (3.10+).

git clone https://github.com/Garyouki/shiftguard && cd shiftguard
python -m unittest discover -s tests -v

# Replay the bundled sample under every controller, then summarize
python scripts/replay_controllers.py examples/sample_trajectories.jsonl out/replayed.jsonl
python scripts/analyze_experiment.py out/replayed.jsonl --output-dir out/analysis
cat out/analysis/summary.md

Reproducing the paper

The frozen trajectories (3,456 main-study proposals from Qwen3-4B, Phi-4-mini and Gemma4-8B, plus the Llama-3.2-3B replication, five-seed extension, prompt-caution control, and end-to-end replanning run) are on the Hugging Face Hub:

pip install huggingface_hub
python scripts/download_data.py            # ~155 MB into ./data

python scripts/replay_controllers.py data/trajectories_v1.jsonl out/main_replayed.jsonl
python scripts/analyze_experiment.py out/main_replayed.jsonl --output-dir out/main_analysis

The replay scores both baseline controllers and the guard family. Its output matches all 3,456 rows of trajectories_v2_guards_replayed.jsonl exactly.

Other drivers:

pip install -e ".[analysis]"
python scripts/evaluate_learned_router.py REPLAYED.jsonl --output-dir ROUTER
python scripts/cost_sensitivity.py REPLAYED.jsonl --output COSTS.csv
python scripts/plot_results.py REPLAYED.jsonl --analysis-dir ANALYSIS --output-dir figures

Collecting new trajectories

Collection needs a local Ollama server. The checkpoint digests used in the paper are recorded in model_lineage.json in the dataset.

python scripts/run_pilot.py --instances-per-template 4 --seeds 13 42 2026
python scripts/run_pilot.py --instances-per-template 4 --seeds 13 42 2026 --models llama3.2:3b
python scripts/run_pilot.py --instances-per-template 4 --seeds 13 42 2026 --prompt-variant aware

Scope

The simulator and the normalized check costs are a controlled research setting. They do not establish production deployment guarantees.

Citation

@misc{zheng2026staledoesnotmeanunsafe,
  title={Stale Does Not Mean Unsafe: Guard Precision for Tool-Using LLM Agents under Infrastructure State Races},
  author={Zihao Zheng and Jiayu Long and Baichuan Li and Junyi Yao},
  year={2026},
  eprint={2609.29522},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  doi={10.48550/arXiv.2609.29522},
  url={https://arxiv.org/abs/2609.29522}
}

License

Code: MIT (see LICENSE). The trajectory dataset has its own card and license on the Hugging Face Hub.

Metadata

Release files for shiftguard 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for shiftguard 0.1.0
File Size Uploaded Provenance Actions
shiftguard-0.1.0.tar.gz 17.1 kB With provenance Details for shiftguard-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for shiftguard 0.1.0
File Interpreter ABI Platform Actions
shiftguard-0.1.0-py3-none-any.whl Python 3 none any Details for shiftguard-0.1.0-py3-none-any.whl

Total release size: 31.5 kB

Release files / shiftguard-0.1.0.tar.gz

Download URL shiftguard-0.1.0.tar.gz
Size 17.1 kB
Tags Source
SHA-256 checksum
How to use checksums
71c57e61c9ac07707507227c0d51c80b7ab596bba88acf032816cebb9b7dc959
BLAKE2b-256 checksum
How to use checksums
52c64902d6b8064875b782fe01333833cc1737992bb1a696052f4053b7bd2381
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.

Transparency log

Release files / shiftguard-0.1.0-py3-none-any.whl

Download URL shiftguard-0.1.0-py3-none-any.whl
Size 14.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
39d9c253b92299d529582e24c4b551a1668e6cefb4f5916e362b63913fb923a2
BLAKE2b-256 checksum
How to use checksums
09d4ef1630869abc83b084e2ec3b5288a215eeeacd96b1a4bf074b19b0ebf11c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.

Transparency log

Security for shiftguard 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for shiftguard-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page