agentic-ci

Run AI coding agents in sandboxed CI environments with streaming output and telemetry. Supports multiple agent harnesses (Claude Code, OpenCode, and Codex) and isolation backends so you can choose the right tradeoff between simplicity and security.

Backends

Local

Runs the agent directly in the current environment with no container or sandbox layer. The agent binary must already be installed and on PATH. Environment variables (auth, OTEL, model) are set automatically by the harness.

Good for: running inside an existing CI container (e.g. a Prow step image) where the agent CLI is pre-installed and an extra isolation layer is unnecessary.

Requires: the selected agent CLI on PATH (for example claude, opencode, or codex).

Podman (default)

Runs the agent inside a Podman container. Each run creates a fresh container that auto-deletes on exit. The work directory is mounted into the container and gcloud credentials are mounted read-only.

Important: The Podman backend provides only basic container-level isolation. It uses --network host, so the agent has unrestricted network access. There is no filesystem sandboxing beyond the container boundary itself and no network policy enforcement. Use the OpenShell backend if you need stronger security controls.

Good for: local development, CI runners that already have Podman, quick one-off runs in trusted environments.

Requires: podman, a container image with the agent CLI installed (e.g. ghcr.io/opendatahub-io/ai-helpers:latest).

OpenShell

Runs the agent inside an OpenShell sandbox with network policy enforcement, Landlock-based filesystem access control, and fine-grained endpoint restrictions. Network policies limit which hosts the agent can reach (e.g. only Vertex AI, GitHub, PyPI) and filesystem policies restrict which paths are writable. An embedded gateway starts per CI job — no external infrastructure required.

Good for: production CI where you need to control what the agent can access on the network and filesystem.

Requires: openshell and openshell-gateway installed on the host. See Local Development with OpenShell to set it up on your workstation.

Install

uv tool install agentic-ci
# OR
pip install agentic-ci

Usage

Run a prompt

# Local (direct execution, no container)
agentic-ci run --backend local "Fix the flaky test in test_auth.py"

# Podman (default backend)
agentic-ci run "Fix the flaky test in test_auth.py" \
    --image ghcr.io/opendatahub-io/ai-helpers:latest

# OpenShell
agentic-ci run --backend openshell "Fix the flaky test in test_auth.py"

Setup and stop

setup creates and starts the sandbox environment. stop tears it down. run auto-calls setup if the sandbox isn't already running.

# Start the sandbox
agentic-ci setup --image ghcr.io/opendatahub-io/ai-helpers:latest

# Run multiple prompts in the same sandbox (use --keep to prevent auto-teardown)
agentic-ci run "Fix the flaky test" --keep \
    --image ghcr.io/opendatahub-io/ai-helpers:latest
agentic-ci run "Update the changelog" \
    --image ghcr.io/opendatahub-io/ai-helpers:latest

# Tear down the sandbox
agentic-ci stop

Options

agentic-ci {setup,run,stop} [options]

Extra arguments after the prompt are passed through to the selected agent CLI.

Examples

# Local backend with extra Claude args (everything after -- is passed through)
# Note: build_args() sets --permission-mode bypassPermissions by default;
# pass --permission-mode default to restrict tools via --allowedTools
agentic-ci run --backend local \
    "Fix the flaky test" \
    -- --permission-mode default --allowedTools "Bash Read Edit" --max-turns 10 --verbose

# Local backend with --continue for multi-stage flows
agentic-ci run --backend local "Summarize your findings" \
    -- --continue --max-turns 5

# Use a specific model
agentic-ci run "Update the changelog" \
    --image ghcr.io/opendatahub-io/ai-helpers:latest \
    --model claude-sonnet-4-6

# Disable parsed stream output (prints raw agent output)
agentic-ci run "Run the test suite" \
    --image ghcr.io/opendatahub-io/ai-helpers:latest \
    --no-streaming

# Disable telemetry
agentic-ci run "Fix lint errors" \
    --image ghcr.io/opendatahub-io/ai-helpers:latest \
    --no-otel

# Run with post-agent gates
export TICKET_KEY=AIPCC-123
export BOT_EMAIL=bot@ci.com
agentic-ci run "Fix the bug" \
    --image ghcr.io/opendatahub-io/ai-helpers:latest \
    --post-gates sensitive-files,commit-author,commit-message-key,gitleaks

# OpenShell with custom policy
agentic-ci run --backend openshell "Deploy staging" \
    --policy custom-policy.yml

# OpenShell with repo-level policy (auto-discovered from
# .agentic-ci/openshell-policy.yml in the workdir)
agentic-ci run --backend openshell "Add input validation"

Gates

Gates validate data before and after an AI agent runs. Pre-gates can block execution early; post-gates validate output to catch dangerous changes. Gates read their configuration from environment variables.

Built-in post-agent gates:

Pre-agent gates are supported via --pre-gates with custom implementations (e.g. filtering by comment domain or author).

All required environment variables are validated before any gate runs. If any are missing, the CLI exits immediately with a clear error listing every missing variable and which gate needs it.

Credentials

Anthropic, Vertex AI, and OpenAI authentication are supported. The auth family follows the selected harness — Claude Code auto-detects Anthropic API key, Claude subscription OAuth token, or Vertex AI, while Codex always uses OpenAI — and the resolved mode is logged at startup.

Anthropic API key (direct)

Set ANTHROPIC_API_KEY in the environment. No gcloud credentials are needed; the key is passed directly to the agent inside the container or sandbox. Vertex-specific env vars and credential mounts are skipped.

export ANTHROPIC_API_KEY=sk-ant-...
agentic-ci run "Fix the bug" --image ghcr.io/opendatahub-io/ai-helpers:latest

Claude subscription (OAuth token)

For individual use, the Claude Code harness can authenticate with a Claude Pro or Max subscription instead of an API key or Vertex AI. Generate a long-lived token with claude setup-token and export it:

export CLAUDE_CODE_OAUTH_TOKEN=sk-ant-oat01-...
agentic-ci run "Fix the bug" --image ghcr.io/opendatahub-io/ai-helpers:latest

ANTHROPIC_API_KEY wins when both are set. The local and podman backends pass the token to the agent the same way as an API key. On OpenShell no provider is created: the sandbox env script exports the token, and the agent can read it from its environment, as with an API key. The token is tied to your subscription and its usage limits, so use it for personal and development runs, not shared CI. OpenCode and Codex do not read it.

Vertex AI (default)

When neither ANTHROPIC_API_KEY nor, for Claude Code, CLAUDE_CODE_OAUTH_TOKEN is set, all backends use Vertex AI for Claude API access via gcloud Application Default Credentials.

The podman backend checks credentials in this order:

1. GCLOUD_CREDENTIALS env var (raw JSON or base64-encoded)
2. GCP_SERVICE_ACCOUNT_KEY env var (file path, raw JSON, or base64-encoded)
3. ~/.config/gcloud/application_default_credentials.json
4. Path in GOOGLE_APPLICATION_CREDENTIALS env var

The openshell backend uploads the local ADC file (~/.config/gcloud/application_default_credentials.json or GOOGLE_APPLICATION_CREDENTIALS) into the sandbox.

OpenAI Codex

For non-interactive Codex runs, set OPENAI_API_KEY for the single job or invocation. An existing login under CODEX_HOME is also supported by the local backend. When an API key is present, agentic-ci uses Codex's non-interactive login --with-api-key flow before executing the prompt.

On the OpenShell backend, Codex runs with --dangerously-bypass-approvals-and-sandbox, the same trust level as Claude Code (bypassPermissions) and OpenCode (--dangerously-skip-permissions): the OpenShell sandbox and its network policy already isolate the agent, and Codex's own approval prompts and workspace-write sandbox would only block skills from reaching allowed hosts. On the podman and local backends (the plain runner image), Codex keeps --approve-for-me, so its inner sandbox and automatic approval review stay active. Ephemeral mode stays off in both cases, so sessions remain available for follow-up turns. Pass additional Codex arguments after -- and before the prompt is sent to Codex, for example:

agentic-ci run --backend local --harness codex \
    "What unique fact did you remember?" -- resume --last

OpenShell currently follows agentic-ci's existing L4 API-key pattern: the real key is written into the sandbox environment and Codex login state. The env script that carries it is deleted as soon as the agent command sources it, but the key remains available to the agent process (and, for Codex, in $CODEX_HOME/auth.json) for the duration of the run. This matches the existing backend behavior but does not provide OpenShell's stronger L7 credential isolation. A future change should migrate API-key providers together to profile-backed L7 inspection so sandboxes receive only opaque placeholders.

export OPENAI_API_KEY=...
agentic-ci run --backend local --harness codex "Fix the bug"

Codex user configuration remains enabled so installed plugins and skills are available. During telemetry-enabled runs, agentic-ci supplies per-run Codex configuration overrides that export logs, metrics, and traces to its local OTLP collector. The completion summary estimates Codex cost from a generated snapshot of LiteLLM's OpenAI price map. Run make update-cost-map to resolve LiteLLM main to its current commit SHA and regenerate the bundled snapshot from that immutable revision. Set AGENTIC_CI_LITELLM_COST_MAP to a LiteLLM-format JSON price map to override the bundled data; unknown models still report token usage but do not produce a dollar estimate.

Environment Variables

Streaming Output

By default, agent output is parsed into human-readable CI logs with:

• Colored ANSI output (thinking in cyan, tool calls in gray)
• Tool call summaries (bash commands, file paths, agent dispatches)
• Token count display with throughput rate
• OTEL token/cost summary at completion

Disable with --no-streaming to skip the parsed output and print raw agent output, or --no-otel to skip the token/cost summary.

Python API

from agentic_ci.backends import create_backend
from agentic_ci.harness import create_harness

harness = create_harness("claude-code")

# Reasoning effort: env var, else the registry default ("high"). Pass the CLI
# flags and the value; the backend exports it to the agent as
# AGENT_REASONING_EFFORT.
effort, subagent_effort = harness.resolve_efforts()
effort_args = harness.build_effort_args(effort, subagent_effort)

# Podman backend
backend = create_backend(
    "podman", harness=harness, workdir="/path/to/repo", image="my-image:latest"
)
backend.setup()
rc = backend.run(
    prompt="Fix the bug", model="claude-sonnet-4-6", extra_args=effort_args, effort=effort
)
backend.stop()

# Local backend (no container)
backend = create_backend("local", harness=harness, workdir="/path/to/repo")
backend.setup()
rc = backend.run(
    prompt="Fix the bug",
    model="claude-sonnet-4-6",
    extra_args=[*effort_args, "--max-turns", "10"],
    effort=effort,
)
backend.stop()

Additional Modules

The package includes several library modules used by downstream pipelines:

• agentic_ci.jira — Jira REST API client with acli delegation, ADF (Atlassian Document Format) conversion, and rate limiting.
• agentic_ci.forge — GitHub/GitLab MR/PR helpers (status, comments, labels: exists / create / attach) plus agentic-ci forge CLI.
• agentic_ci.git — Git operations (clone, branch, push, diff, commit info extraction) with security hardening.
• agentic_ci.pipeline — GitLab child pipeline YAML generation with hash-based slot distribution.
• agentic_ci.verdict — Structured verdict JSON schema validation.
• agentic_ci.telemetry — Generic producer-event validation and OTLP transport for the MLflow export path.

Building a Pipeline with the Generic Skill Runner

agentic-ci provides a generic skill runner framework that any project can use to build its own AI-powered CI pipeline. You define what happens at each stage via callable hooks; the framework handles container execution, retries, OTEL cost tracking, and gate orchestration.

Quick Start

import json
from pathlib import Path
from agentic_ci.skill import SkillConfig, run_skill

config = SkillConfig(
    skill_name="my-review",
    prompt_builder=lambda ticket_key, mode, skill_name, **kw: (
        f"Use the /{skill_name} skill to review ticket {ticket_key}."
    ),
    verdict_loader=lambda work_dir: json.loads((work_dir / "verdict.json").read_text()),
    label_applier=lambda ticket_key, verdict, **kw: print(f"[{ticket_key}] verdict: {verdict}"),
)

rc = run_skill(
    config,
    ticket_key="PROJ-123",
    work_dir=Path("/tmp/work"),
    config_dir=Path("/tmp/config"),
)

SkillConfig Hooks

All domain-specific behavior is injected via hooks on SkillConfig:

Extra Skills (Extension Hooks)

extra_skills lets you configure additional skills that the agent should run at specific hook points during the pipeline (e.g., run a preflight review after implementing a fix).

config = SkillConfig(
    skill_name="autofix-resolve",
    extra_skills=[
        {"name": "preflight", "args": "--local --fix", "hooks": ["post_implement"]},
        {"name": "lint-check"},
    ],
    context_dir=".autofix-context",  # where config.json is written (default: ".context")
)

Each entry is an object with name (required), args (optional), and hooks (optional).

When extra_skills is non-empty, run_skill() writes {context_dir}/config.json before launching the container:

{
  "extra_skills": [
    {"name": "preflight", "args": "--local --fix", "hooks": ["post_implement"]},
    {"name": "lint-check"}
  ]
}

Important: run_skill() only writes the config file — it does not execute the extra skills directly. The orchestrator skill (the one launched by run_skill()) must include instructions in its SKILL.md to read {context_dir}/config.json and invoke each extension at the appropriate hook point. The extra skills themselves don't need any awareness of this file. context_dir is validated to stay within work_dir (rejects path traversal and symlinks).

Pipeline Flow

run_skill() executes this sequence:

1. Pre-gates -- each pre_gates callable can block the run early (returns a message to skip, None to continue)
2. Context -- context_writer writes ticket data and supporting files
3. Extension config -- extension_config_writer sets up Claude plugins/skills
4. Prompt -- prompt_builder produces the prompt string
5. Container -- launches Claude via PodmanBackend (or a custom container_runner)
6. Retry -- transient failures (exit 124/137/143) retry once if mode is in retryable_modes
7. Cost -- parses OTEL metrics from the run directory
8. Post-gates -- each post_gates callable validates the output (e.g. sensitive file check, gitleaks)
9. Verdict -- verdict_loader reads the agent's structured output
10. Report -- label_applier applies labels, posts comments, transitions tickets

Reasoning Effort

Every run passes a reasoning effort to the agent CLI so quality does not depend on the model's own default (Codex, for example, defaults to low on gpt-5.6-sol). The effective value is resolved as --effort flag, then the harness env var (CLAUDE_REASONING_EFFORT, OPENCODE_REASONING_EFFORT, CODEX_REASONING_EFFORT), then the registry default_effort, which is high for all three harnesses. Codex sub-agents follow CODEX_SUBAGENT_REASONING_EFFORT, else the main effort. The value none passes no flag. Invalid values fail before the agent starts. The effective effort is printed at run start and recorded on the synthetic root span as agent.reasoning_effort (and agent.subagent_reasoning_effort).

Every backend exports the effective effort to the agent as AGENT_REASONING_EFFORT, next to AGENT_MODEL, so skills can read the model and effort in use (including the defaults) without knowing the harness. It is unset when none disables the effort flag. Like AGENT_MODEL, it is output only: agentic-ci never reads it, so a run started inside an agent does not inherit the outer run's effort.

Model Routing with run_routed_skill()

run_routed_skill() runs the same pipeline as run_skill() but picks the model per task. Before the skill runs, a short classifier invocation on the harness default model (CLAUDE_MODEL, OPENCODE_MODEL or CODEX_MODEL, else the harness default) rates the task as low, medium or high and writes _run/route.json. The skill then runs on the matching tier, a ModelTier(model, effort) where effort maps to claude --effort, opencode --variant or codex -c model_reasoning_effort=.

from agentic_ci.routing import ModelTier
from agentic_ci.skill import SkillConfig, run_routed_skill

config = SkillConfig(
    skill_name="my-resolve",
    prompt_builder=my_prompt_fn,
    verdict_loader=my_verdict_fn,
    # Optional: override any tier; unlisted tiers keep the harness defaults.
    model_tiers={"low": ModelTier("claude-sonnet-4-5", "low")},
)
result = run_routed_skill(config, ticket_key="PROJ-123", work_dir=work_dir, config_dir=cfg)
print(result.rc, result.route.tier, result.route.model, result.route.source)

Default tiers, defined once in agentic_ci.models.MODEL_REGISTRY together with each harness's default model and accepted effort values (the high tier is always the harness default model):

Behavior:

• The classifier runs inside the same sandbox as the skill (same container, credentials and network policy) and may read files to gauge scope. Its raw stream goes to _run/classifier-output.txt. classifier_max_turns caps it where the CLI supports a turn limit (Claude Code --max-turns).
• Any classifier failure (non-zero exit, missing or invalid route.json) logs a warning and falls back to the default model at the resolved default effort, which is exactly what run_skill() would do. RouteDecision.source is then fallback.
• The decision is made once per call; retries inside run_skill() reuse it.
• force_tier="high" skips the classifier and pins a tier.
• A skill.routed event (tier, model, effort, source) is appended to the run's _run/claude-otel.jsonl. OTEL cost totals include the classifier.
• config.container_runner must be unset; custom runners have no model surface.
• A tier with effort=None runs at the harness default effort (--effort, the effort env var, or the registry default_effort, which is high), the same as an unrouted run.
• OpenCode variant names depend on the model: Claude 4.6 ids accept low, medium, high, max; claude-sonnet-4-5 accepts only high and max.

Example: jira-autofix

The jira-autofix project uses this framework to build an automated Jira bug-fix pipeline:

config = SkillConfig(
    skill_name="autofix-resolve",
    prompt_builder=_build_prompt,  # Jira-specific prompt
    context_writer=_write_context,  # Writes ticket.json to .autofix-context/
    verdict_loader=_load_verdict,  # Reads .autofix-verdict.json
    label_applier=_apply_labels,  # Manages jira-autofix-* labels
    cost_formatter=_format_otel_cost,  # Formats cost for Jira comments
    post_gates=[_autofix_post_gate],  # Commit author check, sensitive files, gitleaks
)

Container Images

Pre-built container images for running AI coding agents in CI are published to quay.io/aipcc/agentic-ci/:

Images are rebuilt daily via GitHub Actions and version-managed by Renovate. See Container Image docs for usage details.

make claude-build              # build Claude Code runner image locally
make opencode-build            # build OpenCode runner image locally
make codex-build               # build Codex runner image locally
make ci-build                  # build CI podman image locally
make openshell-claude-build    # build Claude sandbox locally
make openshell-opencode-build  # build OpenCode sandbox locally
make openshell-codex-build     # build Codex sandbox locally
make openshell-ci-build        # build OpenShell CI image locally

Documentation

API reference documentation is auto-generated from docstrings and published to GitHub Pages.

To build the docs locally:

tox -e docs

Or to preview with live reload:

uv run --with '.[docs]' mkdocs serve