osp-provider-sdk

A provider connects OSP to your service: it declares resource kinds and actions, reads inventory, plans changes, then applies approved work through your backend. A request flows portal → orchestrator → your provider → plan → approval → apply. The orchestrator authenticates callers and manages approvals; your provider owns backend validation, ownership checks and safe effects.

The 5-minute path

Use Python 3.13 or newer and uv. The wheel contains the SDK, not examples/ or their YAML. Example commands below run from a clone; a generated provider includes its own backend and policy.

1. Install: uv venv --python 3.13 then uv pip install osp-provider-sdk==0.6.0.

2. Scaffold (UiO GitHub SSH access and network/VPN required):

   git clone git@github.uio.no:IT-OPS/osp-provider-cookiecutter.git
   uvx cookiecutter ./osp-provider-cookiecutter --no-input
   cd osp-provider-toaster
   

3. Run: make dev, then make demo. The first stack build takes longer than five minutes and needs Podman or Docker and several GB of free image storage.

4. Test offline: make check synchronizes tools and runs lint, types and tests. Preview a plan with uv run python -m osp_provider_sdk plan toaster_provider:app create '{}' --name lunch. This enters your backend lifespan but does not call apply.

See the scaffold guide for prerequisites and the tutorial for the next action. For worked code, clone this repository and read the bucket provider and its backend and policy. The bucket backend is in memory; durable idempotency belongs in your real backend.

The package layers are contracts → runtime → SDK → providers. The SDK is optional; the orchestrator never installs it. See the API contracts, including outcomes.

Ownership is contacts

ResourceOwnership declares contact mailboxes persisted on the resource. The orchestrator derives owning netgroups from those contacts. A backend's domain owner group (for example a Vault group name) is an ordinary provider field, independent of OSP ownership:

ownership = ResourceOwnership(contacts=("storage@example.org",))
fields = {"owner_group": "vault-team-storage"}

Persist ctx.contacts at creation and return them in inventory. Listing filters visibility; resize/destroy must also check current ownership at their entrypoint. Allow scoped Admin administration and use ctx.policy.allows for audited sudo, as the worked examples do. Admin authority does not grant team membership.

Same-team peer approval

For same-team approval, use a peer Gate. For example, in policy.yaml:

request_policy:
  version: 1
  delete_approval: {kind: peer, required_approvals: 1}

This makes destructive actions require one peer approval. Each gated action must declare a read-only .plan. For backend-specific decisions, return Plan(summary, gates=(Gate("review", "peer", message, stable_subject),)). Eligibility uses the task's request netgroup plus current Writer-or-higher provider Role. The requester is excluded from ordinary decisions; scoped Admin may decide without peer membership. Authorized Admin sudo is a separate audited override. The provider never looks up approvers or counts votes. Changed plan fingerprints require fresh approval; see planning and approval.

Bundled policy is the starting point; optional live external policy is described in policy reload.

Invocation and lifetime

app.run() reads RABBITMQ_URL, PROVIDER_INSTANCE, PROVIDER_INSTANCE_ID and CONCURRENCY (default 1); keyword arguments override any runtime option. Queue names come from ProviderIdentity. For example, PROVIDER_INSTANCE=dev python -m examples.buckets starts the example's RabbitMQ runner when RABBITMQ_URL is provided.

To run the bucket commands and direct-call example, first clone the sources:

git clone git@github.uio.no:IT-OPS/osp-provider-sdk.git
cd osp-provider-sdk
uv venv --python 3.13
uv pip install --no-sources -e .
source .venv/bin/activate

A provider's __main__.py is just app.run(). The SDK command covers the rest without an orchestrator; it imports the app and enters lifespan only for an online doctor and for plan:

python -m osp_provider_sdk manifest examples.buckets:app [--check [PATH]]
python -m osp_provider_sdk doctor examples.buckets:app [--offline]
python -m osp_provider_sdk plan examples.buckets:app create '{"quota_gib": 150}' --name logs

plan previews ordinary Writer policy for local provider authors. It does not call apply, but enters lifespan and reads the backend (lifespan may initialize local schema). It grants no live identity or elevation; orchestrator admission still uses the authenticated caller's current Role.

doctor --offline checks declarations and policy. Online it also connects to RABBITMQ_URL and runs the provider's own checks, which may use app.state and return problem strings:

@app.doctor
def data_dir() -> list[str]:
    return [] if DATA_DIR.is_dir() else [f"{DATA_DIR} is missing"]

It prints ok (exit 0) or one problem per line (exit 1). plan prints the planner result an assessment would return, against the configured backend.

Offline invocation uses with app.running():. This validates declarations, enters lifespan, and owns one event loop if any callback or lifespan is async. All async backend clients are created and closed on that loop. Runtime workers block for their handler's result, so runtime concurrency still bounds work. Async handlers must move blocking backend calls into asyncio.to_thread. Sync handlers returning awaitables fail with an instruction to use async def.

from osp_provider_contracts import ProviderRequest, RequestContext
from examples.buckets import app

with app.running():
    result = app.execute(
        ProviderRequest("create", name="logs"),
        RequestContext(
            "local-task",
            netgroup="team-storage",
            contacts=("storage@example.org",),
            role="writer",
        ),
    )
    print(result.produced_resources)

Planning and approval

Assessment, CHECK and APPLY use one planner: model validation → shared rules → custom plan → combination. Assessment always describes APPLY intent and has no cleared approvals. It returns structured rejection, incomplete or approval facts without calling apply. CHECK and no-op plans stop before apply and return {"plan": <planner result>}; an App subclass that must keep an existing CHECK shape overrides plan_result(req, plan) and copies only public facts.

Gated actions must declare .plan and author a concrete summary. The Gate key names a requirement; its fingerprint covers every frozen Gate field except the key, including summary and effective inputs. Use native IDs, before/after values and material destructive scope. Bind destructive work to a content revision or immutable object-version scope, including its fingerprint in public evidence. Apply must enforce that scope with conditional writes and refuse a nonempty bucket delete. Leave out secrets, timestamps and unrelated volatile counts. If the planned work changes, approval must be obtained again. Private Plan.data never enters the public assessment; the checked frozen input model and the same Plan reach apply.

Custom plans must be read-only. Apply cannot request approval: a late ApprovalRequired becomes late_gate_contract_violation with uncertain state. Keep live-state-dependent authorization wholly in the custom plan when shared rules over model defaults would check different values from those executed.

Progress, cancellation and results

Stages use <system>.<verb> keys. Declared stages determine 1-based position; repeat and skip them freely. Progress counts are monotonic within a stage. The first and final counts are sent, intermediate updates are limited to one every two seconds, and the latest suppressed count is flushed on stage exit. Evidence is a JSON-safe mapping under 2 KiB without reserved progress keys. Offline calls log progress. Delivery failures do not fail backend work.

cancellable=True is cooperative. cancel_task accepts a request; only ctx.checkpoint() stops work. Call it only after prior backend operations have settled and the current state is safe to leave. Stage entry and progress never stop work. A request after the last checkpoint can finish normally.

Use Done for ordinary success or return a full ProviderResult for advanced outcomes (pending_retry, provenance, mixed resources). Advanced results retain all their fields. Record each survivor with ctx.created, specifying kind= for mixed kinds, and call ctx.removed only after confirmed cleanup. Survivors are merged into success, typed errors, unexpected failures and cancellation results. There is no rollback or durable SDK journal. A retried task must rediscover its resources through provider-owned backend identity.

Manifest

Fields come from frozen Pydantic models: str, int, float, bool, scalar Literal (an exhaustive enum), list[...] and dict[...], and T | None = None for an optional input without a default. The model states name (Field(alias=...) for dotted wire names), type, required, default, integer ge/le, string pattern, title (label) and description (help). json_schema_extra adds the other contract field keys: unit, group, semantic, accepts and choices. Choices on a Literal relabel or annotate exactly its values (meta, hidden, aliases, …). Choices on any other type are suggestions: values must validate against the field, and type: "enum" or choices_exhaustive: False may be stated, never True. String lists (including string Literal items) use list; other lists use array, and dictionaries use object. Clients submit structured arrays and objects as JSON. Container item and mapping value types remain physically validated by Pydantic; the manifest does not carry their JSON Schema. Annotate ResourceRef for references. The supported declaration subset is:

Declaration	Supported constraints/defaults
str	pattern; literal default
int	integer ge, le; literal default
float, bool	literal default; no bounds
scalar Literal[...]	exhaustive choices; literal default
list[T], dict[str, T]	literal default or Field(default_factory=list) / dict respectively
T | None = None	optional without a declared default
All fields	alias, title, description, supported json_schema_extra keys
Models	frozen, extra, title; input models require frozen=True

Validators, serializers, min_length/max_length, arbitrary factories and nested models fail startup. For nested JSON use list[dict[str, Any]] or dict[str, Any] and validate backend-specific structure in the planner. Containers are physically validated by Pydantic; do not mutate them in planning. Built-in empty container factories are evaluated once per manifest compilation for the declared default.

An input-model field whose wire name is name declares the action's operational name. Clients send it as the top-level Task name, and direct callers use ProviderRequest(name=...); payload.name is rejected for that declaration. The field must be a string. Its required/default/pattern/label/help metadata comes from the model like every other field. The SDK validates the envelope value through that field and exposes the same defaulted value as req.name and in req.inputs. An alias such as Field(alias="name") uses the same binding. hostname is an ordinary separate input. Declare names for custom verbs as well as create; an action without a name field does not ask clients for one. The validated model also supplies policy facts and plan evidence (normalized_request / effective_inputs). That evidence describes prepared work; resubmit the original inputs and top-level name rather than copying the normalized mapping into Task inputs.

Actions take description, target, destructive, stages, risk_hint and confirmation_phrase. Destructive actions always advertise a phrase, by default the upper-cased action name (snapshot_delete → SNAPSHOT DELETE); clients offer destructive actions only with one. Resources take list fields and filters; list fields carry no required, so views stay meaningful. Exact get is optional; without one the SDK walks list cursors until the native reference is found. Undeclared row fields are dropped while downloads, timestamps and other native metadata are retained. Page is the contracts ResourceListResponse: pass its required observed_at freshness timestamp.

The manifest version is the installed osp-provider-<family> distribution's version (family smoke_stack → osp-provider-smoke-stack) unless App(version=...) sets it; code that is not installed advertises 0.0.0.

Multiple resource kinds

Declare kind= on each action when an App offers multiple kinds. Handlers are selected by (resource_kind, action), so both sandwich/create and drink/create can have their own planner and effects. Manifest v3 keeps each action specification inside its resource kind, so repeated names can declare independent fields, targets, stages and policy. Kinds declare min_role (default reader); actions declare min_role (default writer). The orchestrator enforces both cumulative Role floors without granting ownership or policy overrides. An explicit reader action is supported. The SDK also enforces these floors for direct execution and assessment and checks kind floors for list/exact-reference reads. Trusted ctx.role is available to both planners and effects, separate from policy overrides; list callbacks receive access.role. Use explicit resource kinds in Provider requests and plan --kind drink for an ambiguous action.

The local sandwich shop demonstrates create, update and approved delete for sandwiches and drinks, ownership-filtered inventory, three seconds of progress with safe cancellation, and persistent SQLite orders. It never contacts an external service. The two kinds share quantity and preparation instructions; their products and prices differ. The sandwich kind also offers order: one task prepares both products with the same quantity and name, returning two independent produced-resource links. Each can then be updated or deleted through its own kind's actions. Both rows are committed together, and task-derived references make redelivery idempotent.

quote on either kind is a Reader action that returns the price without changing orders. The bundled shop policy owns the ordinary quantity limit once: its maximum of five supplies the form hint and requires one Maintainer approval above it for Writers and Maintainers. An ordinary Admin is exempt from that quantity rule and may administer another owner's order within the shop's provider scope. Writers and Maintainers must match the order's contacts. Selecting an unrepresented acting team or contact still requires sudo at orchestrator admission; Admin does not grant membership. The physical model maximum of twenty applies to every Role. Delete keeps its independent destructive-action approval.

SANDWICH_SHOP_DB selects the state file (default .local/sandwich-shop.sqlite3). OSP_POLICY_SNAPSHOT optionally selects an external policy bundle; otherwise the shop uses its bundled policy. See policy reload for atomic publication and last-good retention. The canonical local stack registers this bundled example through osp-local.json, outside shared release inventory; see the stack instructions.

Everything else

extra= on App, @app.action and @app.resource (merged into its list) adds any remaining key of the contracts ManifestV3, ActionSpec or ListSpec. Unknown keys and keys owned by an SDK argument fail at startup, and the whole document is then checked with validate_manifest_v3.

app = App("nrec", description="NREC provider", extra={"features": ["console.novnc"]})


@app.resource("vm", fields=VmRow, filters=["name"], extra={"page_size_max": 1000})
def vms(q, access) -> Page: ...


@app.action(
    "resize",
    target="resource",
    inputs=Resize,
    risk_hint="Resizing interrupts the VM.",
    extra={"inputs": {"soft_limits": {"flavor": {"allowed_without_force": ["m1.small"]}}}},
)
def resize(req, ctx) -> Done: ...

app.run(**runtime) overrides any RuntimeConfig field, including the environment and queue defaults: concurrency=5 replaces CONCURRENCY, and prefetch_count, control_concurrency or max_attempts work the same way.

Protocol methods the SDK does not declare stay plain methods on an App subclass: get_console, get_history and get_build_log are found by the runtime, and self.call(fn, *args) runs sync or async backend code on the lifespan's loop with self.state.

Hot-loaded data (reloaded choices, points, actions an operator switched off) overrides effective_manifest(), starting from super().effective_manifest(). The SDK validates its result once per call and uses that one snapshot for capabilities, action membership and list field/filter checks. It can hide declared actions and lists or add list fields and filters, never offer an undeclared handler; input models still validate the inputs themselves. Backend ownership decisions and native backend cancellation remain provider-owned.

Development and snapshots

Install the released contracts/runtime dependencies and SDK development tools:

uv venv --python 3.13
uv pip install --no-sources -e '.[dev]'
.venv/bin/ruff check .
.venv/bin/ty check src tests
.venv/bin/pytest
.venv/bin/python -m osp_provider_sdk manifest examples.buckets:app --check examples/manifest.json
.venv/bin/python -m osp_provider_sdk manifest examples.sandwich_shop:app --check examples/sandwich-shop-manifest.json
.venv/bin/python -m build
.venv/bin/twine check dist/*

Generate snapshots with python -m osp_provider_sdk manifest module:app. --check [path] compares exact canonical JSON, defaulting to manifest.json. Snapshot generation checks startup but never enters lifespan or connects a backend.

CI runs the SDK and shared provider conformance suite with lowest-direct and highest dependency resolution, checks the manifest, builds both artifacts, and imports the installed wheel outside the source directory. Tag CI attaches artifacts to a GitHub release; it does not publish to PyPI.

Releases

Publish reviewed contracts 0.13 first, then refresh and review the runtime lock and publish runtime 0.12.1. Refresh and review the SDK lock before publishing SDK 0.6. Consumer locks must use published registry packages, with no local source overrides. Verify candidate wheels together in a clean environment before publication. Coordinate wire 9 and manifest v3 activation separately from publishing these shared libraries.

The private repository's Actions policy must allow its configured actions. Set up the new distribution name and PyPI credentials or trusted publishing. After review, quality checks and signed release/tag verification, a maintainer can run bash scripts/publish.sh (or -r testpypi). --tag additionally creates, verifies and pushes a signed tag after upload; normal development never runs it. No release or live environment is needed for the local tests.

Scoped sudo policy execution

PlanCtx.policy and Ctx.policy share the runtime's request-scoped Policy owner. It consumes only the orchestrator-authorized RequestContext.sudo from wire 9. Use ctx.policy.allows("stable-rule", condition, evidence=...) for a provider policy judgement; raise the original error when a backend operation cannot proceed. Do not put elevation switches in an input model.

The SDK applies that owner before shared reject-level policy can stop planning and again to provider-authored Plan policy violations/Gates. Assessment reports an allowed plan with override warnings; execution still prepares and applies the real operation. Violation.invalid_value, Violation.missing_required and incomplete execution plans remain failures. Use these only for essential execution data/capability; express configurable business limits as policy.

Execution retains collected policy_overrides on both result data and error extra, including planner rejections and cancellation results. The actual outcome, error detail, retry semantics and surviving resources remain intact. A scheduled future Writer request is ordinary unless the orchestrator separately authorizes its sudo context; a prior management session is not such a grant.

Advisory availability snapshots

Use the existing App.effective_manifest() hook for runtime availability. Start with super().effective_manifest() (a fresh copy), read one supply snapshot, and set availability on a field choice or resource-local action. No restart is needed. Each hint has state (ok, low, out) and RFC 3339 as_of; remaining and a short note are optional. Static declarations can use action extra={"availability": hint} or a field's json_schema_extra choice metadata. The SDK validates the snapshot but does not use advisory hints as policy.

The local shop demonstrates the same hook for both choices and actions. It persists stock in its existing SQLite database, starts each product at five units, reports low at two and out at zero, and rejects insufficient supply in planning and atomically during apply. Decreasing an order quantity returns the difference; deleting a prepared order does not restock consumed ingredients. Composite orders consume both products in one transaction. Redelivery does not consume stock twice.

SANDWICH_SHOP_STOCK optionally configures initial counts, for example {"sandwich":{"Cheese sandwich":4,"Tomato sandwich":3},"drink":{"Water":6}}. Counts seed only unseen stock rows; restarting preserves remaining stock and orders. Existing local orders retain their original default product.

SDK 0.6.0 installs published contracts 0.13 and runtime 0.12.1 from PyPI. The committed lock uses registry packages without sibling source overrides.

Local testing against a real backend

Run the real service in throwaway dev mode, such as Vault server -dev with a local root token, as the OSP local stack does. Point only your test provider at that disposable instance. Keep live tests opt-in (for example OSP_LIVE_TESTS=1 OSP_ALLOW_WRITES=1 checked by your test harness); explicitly enable effects/writes, create known test resources and clean them up. These are provider-owned test flags, not SDK settings. Offline plan still enters lifespan and reads the configured backend. Never use production credentials for this path.