GitInject

A framework for evaluating prompt injection in real AI-powered CI/CD workflows.

📚 Documentation · 📄 Paper · 🧪 Reproduce paper attacks · 🤝 Contributing

🎉 Accepted to the NeurIPS 2026 Evaluations & Datasets Track!

Our paper, GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines, introduces the framework and studies attacks against AI-powered GitHub workflows.

GitInject provisions repositories, installs agent workflows, triggers scenario inputs, and evaluates the resulting repository state and agent output. Use it to reproduce attacks, compare workflow defenses, and build custom experiments.

• Live workflow evaluation: run utility tasks and prompt injection scenarios in real GitHub Actions workflows.
• Independent measurements: track task completion, security breaches, and verified agent invocation. Verdicts can be true, false, or unknown; execution and evaluation errors are reported separately. A security breach means the attack succeeded.
• Inspectable results: each GitHub attempt records copied inputs, execution phases, evidence, and results.
• Attack discovery: the scanner generates and ranks attack hypotheses, then validates candidates through the same live run engine.

A GitLab runner is also available with a narrower execution and evidence contract. See metrics and evidence for how verdicts are determined.

GitInject creates public repositories, installs credentials, triggers real workflows, and deletes repositories during cleanup. Use a dedicated testing account.

Get started

Install Python 3.13+, uv, and the GitHub CLI. Install a released version from PyPI into an existing project:

uv add gitinject
uv run gitinject list workflows
uv run gitinject list scenarios
uv run gitinject run-suite --workflow-labels codex --scenario-type benign --dry-run

For a standalone CLI, use uv tool install gitinject, then run gitinject directly. Python extensions import from gitinject, for example from gitinject.runner import BenchmarkRunner.

To install from Git, use uv add gitinject --git https://github.com/ceferisbarov/GitInject.git. To develop GitInject or reproduce the paper with the checked-in dependency lockfile, use a checkout:

git clone https://github.com/ceferisbarov/GitInject.git
cd GitInject
uv sync --locked
uv run gitinject list workflows
uv run gitinject list scenarios
uv run gitinject run-suite --workflow-labels codex --scenario-type benign --dry-run

The dry run lists compatible pairs without creating repositories or calling models. Configure your GitHub identity and workflow/judge credentials using the installation guide, then run a first benign trial:

uv run gitinject run --workflow codex-pr-review --scenario vulnerable_code_review

This pair needs OPENAI_API_KEY for Codex and GEMINI_API_KEY for semantic evaluation, plus local GitHub authentication. See the quickstart for interpreting results.

Guides and reference

• Run benchmarks, suites, and bundled paper attacks
• Author Python scenarios and research experiments
• Add workflows
• Generate and optimize attacks
• Scan workflows
• Inspect and reproduce results
• CLI reference and Python API

For local documentation previews, strict builds, and GitHub Pages deployment, see documentation development.

Repository layout

Citation

@article{isbarov2026gitinject,
      title={{GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines}}, 
      author={Jafar Isbarov and Umid Suleymanov and Ilia Shumailov and Murat Kantarcioglu},
      year={2026},
      eprint={2606.09935},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2606.09935}, 
}

Contact Jafar Isbarov at isbarov at vt dot edu.