pennsieve-ai-utils

Shared utilities for the Pennsieve AI Co-Scientist stages:

1. pennsieve-ai-scout
2. pennsieve-ai-hypothesize
3. pennsieve-ai-analyze
4. pennsieve-ai-write

Each stage used to carry its own copy of the blackboard schema, the governor LLM client, the model-role resolver, the agent helpers and the literature tools. They now import them from here. The package is published to PyPI as pennsieve-ai-utils (same release flow as pennsieve-llm: push a vX.Y.Z tag), so the Pennsieve App Store build of each stage can pip install it without GitHub credentials.

See RUNBOOK.md for local stage supervision, stopping, review bundles and the remaining Pennsieve Test deployment work.

pip install pennsieve-ai-utils

What's here

Stage-specific science — pipelines, prompts, Discover/DANDI/OpenNeuro inspection, notebook execution, manuscript rendering — stays in each app.

Blackboard contract

from pennsieve_ai_utils import Blackboard

bb = Blackboard.load("blackboard.json")
bb.record_model_assignment(
    stage="hypothesize",
    role="reasoning",
    assignment={"model_id": "us.anthropic.claude-sonnet-4-6"},
)
bb.save("blackboard.json")

• One superset schema carries fields produced by every stage.
• schema_version enables explicit migrations; documents without a version are treated as legacy version 0.
• Unknown fields are preserved at their original object level on a load/save round trip.
• Model provenance is cumulative by stage (model_assignments[stage][role]) rather than overwritten by the next application.

LLM client

from pennsieve_ai_utils.llm import LLMClient
from pennsieve_ai_utils.agents import call_json

client = LLMClient(stage="hypothesize", roles=("reasoning", "synthesis"))
assignments = client.preflight()          # raises ModelNotAvailable if a floor can't be met
for role, assignment in assignments.items():
    bb.record_model_assignment(stage="hypothesize", role=role, assignment=assignment)

verdict = call_json(client, agent="critic", system=..., user=..., run_id=bb.run_id,
                    model="reasoning")

Agents address models by role, never by vendor tier. The legacy names opus / sonnet / haiku still resolve as aliases of reasoning / synthesis / bulk, and VP_MODEL_<ROLE> pins keep working.

Preflight asks the governor's GET /v1/models for the allow-list first and falls back to tripping a model_not_allowed 403 on older governors. Each assigned model is then verified with a one-token call so an IAM block or a retired Bedrock model fails at startup rather than twenty minutes in.

The client needs LLM_GOVERNOR_FUNCTION_NAME (platform-injected) or PENNSIEVE_LLM_MOCK=1 for offline tests — see LLM access on compute nodes.

Processor helpers

from pennsieve_ai_utils.processor import (
    ProcessorEnv, install_sigterm_handler, load_upstream_blackboard, write_outputs,
)

install_sigterm_handler()
env = ProcessorEnv.from_environ()
bb = load_upstream_blackboard(env.input_dir / "blackboard.json",
                              stage="analyze", upstream="Hypothesize",
                              run_id=env.execution_run_id)
...
write_outputs(bb, env.output_dir, client.usage_log)

These follow the Pennsieve processor contract: read from INPUT_DIR, write to OUTPUT_DIR, exit non-zero on failure.

Releasing

# bump version in pyproject.toml, commit, then:
git tag v0.2.0 && git push origin v0.2.0

.github/workflows/publish-pypi.yml builds and publishes via PyPI Trusted Publishing. One-time setup on pypi.org: add this repository as a trusted publisher for the pennsieve-ai-utils project (GitHub environment pypi). Apps pin pennsieve-ai-utils>=0.2,<0.3 and pick up patch releases on rebuild.

Development

python -m pip install -e ".[dev]"
python -m ruff check src tests
python -m pytest