pythonllmhook

Русская версия

pythonllmhook converts explicitly allowed runtime failures into verified Python handlers or source patches. The import name and CLI command are llmhook.

The package is not a repository-wide coding agent. A decorator defines the activation point, runtime evidence, specification, editable files, checks, persistence, and Git policy.

Status: alpha. Use it in a controlled repository. Review generated code before deployment.

Requirements

• Python 3.11 or later
• Git for source evolution
• An OpenAI API key for the OpenAI provider

Installation

python -m pip install pythonllmhook

Install provider and keyring support:

python -m pip install "pythonllmhook[openai,keyring]"

Development checkout:

git clone https://github.com/averagedigital/pythonllmhook.git
cd pythonllmhook
python -m venv .venv
. .venv/bin/activate
python -m pip install -e ".[dev]"

Initialization

Run this inside a Git repository:

llmhook init
llmhook doctor

init creates .llmhook/prompts, adds runtime paths to .gitignore, and appends [tool.llmhook] configuration when it is absent. It does not replace an existing section.

Credentials

Environment variables are the default for CI and containers:

export OPENAI_API_KEY="..."

Local keyring commands:

llmhook auth login openai
llmhook auth login openai --profile work
llmhook auth status
llmhook auth logout openai

Credential lookup order: explicit provider value, provider environment variable, LLMHOOK_API_KEY, OS keyring. The auth CLI writes keys only to the OS keyring. The library does not intentionally copy resolved credentials into project files, incidents, reports, or logs.

Runtime handler

from llmhook import llm_except


@llm_except(
    spec="""
    Accept integers from 0 to 100.
    Convert digit-only strings to int.
    Reject other values.
    """,
    exceptions=(TypeError, ValueError),
    returns=int,
    checks=["pytest tests/test_score.py -q"],
    execution="apply_handler",
    persistence="generated_module",
)
def score(value: object) -> int:
    if not 0 <= value <= 100:
        raise ValueError("invalid score")
    return value

On a supported failure, the provider returns matches and handle functions. llmhook parses the code, rejects denied imports and calls, replays the failing input in a subprocess, runs configured checks, and then activates the handler. Active handlers are ordinary Python modules under .llmhook/generated. A later matching input does not call the model.

Execution modes:

• raise: record or generate, then raise the original exception.
• apply_handler: return the handler result.
• retry_function: require RetryInput, then call the original function once.

Persistence modes:

• none: current call only.
• memory: current process only.
• generated_module: .llmhook/generated.
• source_patch: use the evolution pipeline.

Source evolution

from llmhook import llm_evolve


@llm_evolve(
    spec="Convert digit-only score strings to int. Reject all other strings.",
    exceptions=(TypeError, ValueError),
    context=["tests/test_score.py"],
    editable=["src/app.py:10-30", "tests/test_score.py"],
    checks=["ruff check src tests", "pytest tests/test_score.py -q"],
    mutation="local_branch",
)
def normalize_score(payload: dict[str, object]) -> int:
    score = payload["score"]
    if not 0 <= score <= 100:
        raise ValueError("invalid score")
    return score

local_branch creates a detached worktree at the incident commit, validates the diff against editable, applies it, runs checks, creates llmhook/<hook>-<fingerprint>, and commits there. The caller's branch and working tree are not changed. The default does not push and does not create a pull request.

Mutation modes:

• none: verify and report only.
• local_source: apply a verified patch to the current working tree without a commit.
• local_branch: create a local branch and commit in a worktree.
• push_branch: also push; requires permissions.push_branch = true.
• pull_request: also create a draft PR with gh; requires push and PR permissions.
• live_source: apply to the running application's source tree. It requires both LLMHOOK_ALLOW_LIVE_MUTATION=1 and permissions.live_source_mutation = true.

Source changes take effect after process restart unless the application supplies its own reload mechanism.

Configuration

[tool.llmhook]
mode = "development"
max_attempts = 3
max_context_bytes = 150000

[tool.llmhook.runtime_model]
provider = "openai"
model = "MODEL_NAME"

[tool.llmhook.git]
mutation = "local_branch"
branch_prefix = "llmhook/"

[tool.llmhook.permissions]
call_runtime_model = true
generate_handler = true
generate_patch = true
run_commands = true
push_branch = false
create_pr = false
live_source_mutation = false

[tool.llmhook.checks]
commands = ["ruff check src tests", "pytest -q"]
timeout_seconds = 300

Modes:

• off: call the original function only.
• capture: store incidents and do not call a model.
• development: allow configured generation and mutation.
• production: follow explicit permissions.

LLMHOOK_DISABLE=1 disables capture, model calls, and mutation. Decorated functions still run.

Environment overrides: LLMHOOK_MODE, LLMHOOK_MODEL, LLMHOOK_PROVIDER, LLMHOOK_MUTATION, LLMHOOK_ALLOW_LIVE_MUTATION, and LLMHOOK_DISABLE.

CLI

llmhook incidents list
llmhook incidents show INCIDENT_ID
llmhook incidents ignore INCIDENT_ID
llmhook handlers list
llmhook handlers show HOOK_ID
llmhook handlers disable HOOK_ID VERSION
llmhook repair INCIDENT_ID
llmhook replay INCIDENT_ID
llmhook evolve HOOK_ID
llmhook config show

CI

Use environment credentials. Do not enable push, pull request, or live mutation permissions in a test job. Run:

python -m pytest -q
ruff check src tests
pyright src tests
python -m build

Security limits

Generated handlers and patches are untrusted code. AST checks, subprocess replay, path validation, and tests reduce risk. They do not provide a security sandbox. The verification process can access the current user account and files allowed by the operating system.

Redaction uses field names, length limits, and bounded serialization. It can miss secrets in free text, source files, custom objects, or encoded values. Do not include credentials in decorated function arguments or source context.

Line-range checks validate diff hunk positions against the original file. File boundaries are strict. Complex line movement can be rejected. Review every source patch.

live_source changes files used by a running process. It does not reload the process, guarantee correctness, or provide rollback.

Storage

.llmhook/
  incidents/    deduplicated runtime records
  generated/    handler modules and metadata
  reports/      evolution reports
  prompts/      local prompt additions
  worktrees/    isolated Git worktrees

Incident consistency and locks are local to one machine. There is no distributed coordination.

License

MIT. See LICENSE.