OSA Python SDK
The developer toolkit for the Open Science Archive — define metadata schemas, write validation hooks, build ingesters, and deploy conventions.
Pre-release — APIs will change without notice.
Install
pip install osa-py
Quickstart
A convention defines what data your archive accepts. Here's an example that archives protein structures and runs analysis on the deposited files:
from datetime import date
import httpx
from pydantic import BaseModel
from osa import (
Schema, Field, Record, Reject, hook, convention, Example,
IngesterContext, IngesterRecord,
)
# 1. Define the metadata schema
class PDBStructure(Schema):
__schema_id__ = "pdb-structure"
pdb_id: str
title: str
method: str
resolution: float | None = Field(default=None, unit="Å")
deposition_date: date
molecular_weight: float = Field(unit="kDa")
chain_count: int
# 2. Validate incoming records
@hook
def validate_structure(record: Record[PDBStructure]) -> None:
if record.metadata.resolution and record.metadata.resolution > 4.0:
raise Reject("Resolution too low for reliable analysis")
if not record.files.glob("*.cif"):
raise Reject("At least one CIF file is required")
# 3. Extract features from the data
class Pocket(BaseModel):
pocket_id: int
score: float
volume: float
@hook
def find_pockets(record: Record[PDBStructure]) -> list[Pocket]:
cif = record.files["structure.cif"]
size_kb = cif.size / 1024
return [Pocket(pocket_id=0, score=round(size_kb / 100, 2), volume=size_kb)]
# 4. Pull records from an external source
class PDBIngester:
name = "pdb-rcsb"
schedule = None
initial_run = None
max_file_mb = 50.0
async def pull(self, *, ctx: IngesterContext, limit=None, session=None):
# Continuation comes from `session`: the server hands back whatever the
# previous batch set. Read your start position out of it, and set no
# session on the final batch — that is what ends the run.
ids = ["4TOS", "1TIM", "6LU7"]
start = int((session or {}).get("offset", 0))
window = ids[start : start + limit] if limit is not None else ids[start:]
if start + len(window) < len(ids):
ctx.set_session({"offset": start + len(window)})
for pdb_id in window:
entry = httpx.get(f"https://data.rcsb.org/rest/v1/core/entry/{pdb_id}").json()
file_ref = await ctx.add_file(
pdb_id, "structure.cif",
url=f"https://files.rcsb.org/download/{pdb_id}.cif",
)
yield IngesterRecord(
source_id=pdb_id,
metadata={"pdb_id": pdb_id, "title": entry["struct"]["title"], ...},
files=[file_ref],
)
# 5. Register the convention
convention(
title="Protein Structures",
description="Experimentally determined protein structures with pocket analysis.",
version="1.0.0",
schema=PDBStructure,
ingester=PDBIngester,
hooks=[validate_structure, find_pockets],
files={"accepted_types": [".cif", ".pdb"], "max_count": 5},
# Docs are mandatory, and an agent reads them to decide whether this
# archive can answer a question at all: a purpose, at least one worked
# example, and three distinct trigger questions (a worked example's
# question counts toward the three).
purpose=(
"Look up protein structures by experimental method, resolution and "
"molecular weight, and the binding pockets found in their coordinates."
),
examples=[
Example(
question="Which X-ray structures resolve better than 2 A?",
query="POST /api/v1/data/pdb-structure@1.0.0/records",
interpretation=(
"Filter method = 'X-RAY DIFFRACTION' and resolution < 2.0; "
"resolution is in angstroms and lower is better."
),
)
],
example_questions=[
"What is the molecular weight of 4TOS?",
"Which structures have the largest binding pockets?",
],
when_not_to_use="Not a sequence-search service, and it holds no predicted structures.",
)
Register the convention as a setuptools entry point in pyproject.toml:
[project.entry-points."osa.conventions"]
my_convention = "my_package"
Testing
Test the full convention pipeline end-to-end — the ingester pulls real data, then every hook runs against each record:
osa test --limit 2
Running ingester pdb-rcsb...
Fetched 2 record(s) (4TOS, 1TIM)
Running hooks...
4TOS
✓ validate_structure
✓ find_pockets → 1 Pocket(s)
1TIM
✓ validate_structure
✓ find_pockets → 1 Pocket(s)
2 record(s), 2 accepted, 0 rejected
You can also test individual hooks in-process with synthetic data:
from osa.testing import run_hook
run_hook(validate_structure, meta={"pdb_id": "TEST", ...}, files=tmp_path)
If your source has more records than fit in one batch, test the pagination
too. run_ingester runs a single batch, so it cannot tell you whether your
continuation works — one batch always looks fine.
run_ingester_batches drives the loop the way the server does, threading each
batch's session into the next, and raises StalledIngesterError if a batch
hands back the session it was given and re-emits the same records:
from osa.testing import run_ingester_batches
# PDBIngester has three PDB IDs, so a batch size of one drives three batches.
result = run_ingester_batches(PDBIngester, batch_size=1)
assert len(result.records) == 3
assert len(result.batches) == 3
Local development
Run a full OSA stack locally with Docker:
osa init my-archive
cd my-archive
osa start
This scaffolds a project directory with osa.yaml, .env, and a Docker Compose stack (Postgres, OSA server, docker-socket-proxy). Authentication is handled automatically — osa start mints a dev JWT so osa deploy and osa ingestion start work immediately.
Deploy
Deploy conventions to a running archive:
osa deploy
This builds OCI images for your hooks and ingesters, pushes them to the server's registry, and registers the convention.
CLI
Local instance
| Command | Description |
|---|---|
osa init [dir] |
Scaffold a new project (osa.yaml, .env, .gitignore) |
osa start |
Start the local OSA stack |
osa stop |
Stop the local stack |
osa logs [-f] [service] |
View container logs |
osa status |
Show running containers |
Convention workflow
| Command | Description |
|---|---|
osa test [--limit N] |
Test the full pipeline: ingester → hooks |
osa deploy |
Build OCI images and register conventions |
osa meta |
Print the convention manifest as JSON |
osa ingestion start |
Trigger an ingestion run |
Authentication
| Command | Description |
|---|---|
osa login |
Authenticate via ORCID device flow |
osa logout |
Remove stored credentials |
osa link --server <url> |
Link project to a remote archive |
Concepts
Schema — a Pydantic model defining typed metadata fields. Each field can carry units and constraints. The schema is the contract between depositors, hooks, and the archive.
Hook — a pure function decorated with @hook that receives a Record[T] and returns structured results. The return annotation defines the feature table: -> Model is one row per record, -> list[Model] is many, and either may be | None for a hook that sometimes has nothing to contribute. -> None is a pure validator with no table. Anything else is rejected when the hook is defined. Hooks run as OCI containers — the SDK builds the image, the server orchestrates execution.
Convention — a bundle of a schema, hooks, file requirements, and an optional ingester. Conventions are the unit of deployment.
Ingester — an async generator that pulls records from external systems (APIs, databases, file servers) into the archive on a schedule. Each ingester gets its own image, and osa deploy bakes the ingester's name into it as OSA_INGESTER_NAME, so a package with several conventions runs the right ingester for each.
Continuation — how an ingester paginates. The server calls pull() once per batch and hands back whatever the previous batch passed to ctx.set_session(...), starting with None. So read your start position out of the session argument, make sure the state you set advances, and set no session on the final batch — that is what ends the run. An ingester that hands back the state it was given would pull the same window forever, so the server aborts the run instead.
Record[T] — generic container binding a schema type T to its metadata, files, and SRN (Scientific Resource Name).
License
Metadata
Release files for osa-py 0.10.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| osa_py-0.10.0.tar.gz | 118.6 kB | Without provenance | Details for osa_py-0.10.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| osa_py-0.10.0-py3-none-any.whl | Python 3 | none | any | Details for osa_py-0.10.0-py3-none-any.whl |
Total release size: 192.6 kB
Release files / osa_py-0.10.0.tar.gz
| Download URL | osa_py-0.10.0.tar.gz |
|---|---|
| Size | 118.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
572a751324f2e616f201066acc77dc8d7277a5d75e4cd706803d005c29765fe6
|
|
BLAKE2b-256 checksum How to use checksums |
37a643e199a0a4815629a43b06571b7b11eef5aafebf7668e7d64884846045b1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / osa_py-0.10.0-py3-none-any.whl
| Download URL | osa_py-0.10.0-py3-none-any.whl |
|---|---|
| Size | 74.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7c1b3e644598c234b66c45b199d074b0522be5937d7f1ab6bbb0f446e5fe44d3
|
|
BLAKE2b-256 checksum How to use checksums |
7fedba668e8cbbbd515daf1ba114c82e79af52f3114addf46e02a98c783ec644
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for osa-py 0.10.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.