Skip to main content

pyconveyor

Deterministic YAML pipeline engine for structured LLM extraction.

PyPI Python CI License: MIT

pyconveyor lets you build reliable LLM extraction pipelines by declaring them in YAML. It handles prompt rendering, schema validation, self-correcting retries, parallel steps, batch processing, and benchmarking — so your code handles the domain logic, not the plumbing.


Install

pip install pyconveyor

A simple pipeline

Start with a single LLM step that extracts structured data from a scientific paper. Declare what you want in YAML — no Python required.

# pipeline.yaml
models:
  default:
    provider: openai_compat
    api_key: ${OPENAI_API_KEY}
    model: gpt-4o-mini
    timeout: 120

steps:
  - name: extract
    type: llm
    model: default
    prompt: prompts/extract.j2
    schema:
      title: str
      authors: list[str]
      key_findings: list[str]
{# prompts/extract.j2 #}
Extract structured metadata from the following scientific paper.

Paper:
{{ ctx.paper }}

Return a JSON object with:
- "title": the paper title exactly as written
- "authors": list of author names
- "key_findings": up to 5 key findings as short sentences
pyconveyor run pipeline.yaml --input '{"paper": "Deep learning has revolutionized..."}'

That's it. pyconveyor calls the model, validates the output matches your schema, and retries automatically if the model returns something that doesn't fit.


Bootstrapping a project

Use pyconveyor init to scaffold a working project in one command:

pyconveyor init my_project/ --interactive
cd my_project/
export OPENAI_API_KEY=sk-...
pyconveyor run pipeline.yaml --input '{"paper": "..."}'

The interactive mode asks what you're extracting, which fields you need, and which provider to use. It generates pipeline.yaml, prompt templates, and editor autocomplete config — ready to run.

pyconveyor init my_project/          # static layout with schemas.py
pyconveyor init my_project/ --interactive   # guided setup, inline schema

Rich field descriptions

Add descriptions to your schema fields and they appear automatically in a {{ schema_hint }} variable you can place in any prompt. pyconveyor builds a plain-English field listing for you — no more copying field docs between schema and prompt.

steps:
  - name: extract
    type: llm
    model: default
    prompt: prompts/extract.j2
    schema:
      title:
        type: str
        description: "Paper title exactly as written, including subtitle if present."
      authors:
        type: list[str]
        description: "All author names in order. Include affiliation superscripts if present."
      doi:
        type: str | None
        description: "DOI if listed. Null if not found."
        pattern: "^10\\.[0-9]{4,}/.+$"
      publication_year:
        type: int
        description: "Four-digit year of publication."
{# prompts/extract.j2 #}
Extract structured metadata from the following paper.

{{ schema_hint }}

Paper:
{{ ctx.paper }}

The {{ schema_hint }} renders as something like:

Return a JSON object with the following fields:

- **title** (str, required) — Paper title exactly as written, including subtitle if present.
- **authors** (list[str], required) — All author names in order. Include affiliation superscripts if present.
- **doi** (str | None) — DOI if listed. Null if not found.
- **publication_year** (int, required) — Four-digit year of publication.

You can also add pattern, min_length, max_length, min_items, and max_items constraints. Fields that fail constraints trigger a retry by default, or you can set on_fail: null to silently coerce invalid values to None, or on_fail: warn to log and continue.


Multiple steps

Pipelines grow naturally. Each step's result is available to later steps as {{ steps.name }}.

steps:
  - name: extract
    type: llm
    model: default
    prompt: prompts/extract.j2
    schema:
      title: str
      abstract: str
      methods: list[str]

  - name: classify
    type: llm
    model: default
    prompt: prompts/classify.j2
    schema:
      field: str
      subfield: str | None
      confidence: float
{# prompts/classify.j2 #}
Classify this paper into a research field based on its title and abstract.

Title: {{ steps.extract.title }}
Abstract: {{ steps.extract.abstract }}

Return:
- "field": the primary research field (e.g. "materials science", "molecular biology")
- "subfield": more specific subfield if identifiable
- "confidence": your confidence 0.0-1.0

Steps run in declaration order. A step can reference any prior step's output. The runner returns a RunContext with every step result, attempt logs, and timing.


Controlled vocabularies

Constrain a field to a known set of terms. pyconveyor normalises fuzzy matches and captures novel values for review.

Define your vocabularies as YAML files in a vocabularies/ directory:

# vocabularies/organism.yaml
known:
  - Escherichia coli
  - Saccharomyces cerevisiae
  - Bacillus subtilis
  - Pseudomonas aeruginosa
  - Staphylococcus aureus
label: organism
growth_policy: auto    # auto-approve close matches

Reference them on schema fields by filename:

steps:
  - name: extract
    type: llm
    model: default
    prompt: prompts/extract.j2
    schema:
      organism:
        type: str
        description: "Primary organism studied."
        vocab: organism        # loads vocabularies/organism.yaml
      strain:
        type: str | None
        description: "Strain designation if reported."

Or define a small vocabulary inline — useful for ad-hoc constraints:

schema:
  study_type:
    type: str
    description: "Type of study conducted."
    vocab:
      terms:
        - in vitro
        - in vivo
        - in silico
        - clinical trial
        - field study

When the model returns "E. coli" instead of "Escherichia coli", pyconveyor normalises it automatically. When it returns a genuinely new organism, the value is captured as a suggestion. The {{ vocab_hints }} variable injects known terms into your prompt so the model knows the preferred vocabulary.

Review pending suggestions from the CLI:

pyconveyor vocab review

Self-correcting retries

When a model returns output that doesn't match your schema, pyconveyor feeds the errors back to the model and lets it try again.

steps:
  - name: extract
    type: llm
    model: default
    prompt: prompts/extract.j2
    schema:
      title: str
      authors: list[str]
      doi:
        type: str | None
        pattern: "^10\\.[0-9]{4,}/.+$"
    max_attempts: 3        # give the model up to 3 tries

If the model returns a malformed DOI on the first attempt, the second attempt receives:

Your previous response failed schema validation. Here is what you returned:

{"title": "A Study of...", "authors": [...], "doi": "doi:10.1234/abc"}

Validation errors:
- doi: String must match pattern ^10\.[0-9]{4,}/.+$

Please fix these issues and return a corrected JSON object.

This works for both schema validation errors and JSON parse errors. You control which error types trigger retries with retry_on, cap the feedback size with max_feedback_tokens, and provide custom error templates with error_template.


Batch processing

Process hundreds of papers through the same pipeline with configurable parallelism:

pyconveyor batch pipeline.yaml --input papers.jsonl --output results.jsonl --workers 8
from pyconveyor import BatchRunner

runner = BatchRunner("pipeline.yaml", max_workers=8)
for paper_id, result in runner.run(papers):
    if not result.failed:
        save(result.steps["extract"].value)

Benchmarking

Measure extraction accuracy against a set of known-correct cases:

# Create a benchmark case
mkdir -p benchmarks/paper_001
cat > benchmarks/paper_001/input.yaml << 'EOF'
paper: "Smith et al. (2024) demonstrate that CRISPR-Cas9..."
EOF
cat > benchmarks/paper_001/expected.yaml << 'EOF'
extract:
  title: "CRISPR-Cas9 Applications in Gene Therapy"
  authors: ["J. Smith", "A. Chen", "M. Patel"]
EOF

# Run the benchmark
pyconveyor benchmark benchmarks/ --pipeline pipeline.yaml --report report.html

Compare two pipeline versions side by side, get per-field accuracy scores, and generate HTML reports with charts and Mermaid graphs. Supports YAML and JSON benchmark files, large inputs via $file references, $ignore sentinel to exclude fields from scoring, $ordered for positional list matching, and PDF export.


Ensemble — multi-model consensus

Run multiple models in parallel and auto-merge their outputs:

steps:
  - name: extract
    type: ensemble
    schema: schemas:PaperMetadata
    prompt: prompts/extract.j2
    members:
      - model: gpt4o
      - model: claude
        required: false         # pipeline continues if this model fails
    judge:
      model: gpt4o              # reviews all outputs, returns merged result
      condition: all_succeeded

Member results are accessible individually as steps.extract.gpt4o and steps.extract.claude. If the judge is skipped or fails, the first succeeded member's result is returned.


Schema files and code reuse

As pipelines grow, you can move your schemas to a schemas.py file:

# schemas.py
from pydantic import BaseModel

class PaperMetadata(BaseModel):
    title: str
    authors: list[str]
    doi: str | None
    publication_year: int

class Classification(BaseModel):
    field: str
    subfield: str | None
    confidence: float

Reference them in your pipeline:

steps:
  - name: extract
    type: llm
    model: default
    prompt: prompts/extract.j2
    schema: schemas:PaperMetadata

  - name: classify
    type: llm
    model: default
    prompt: prompts/classify.j2
    schema: schemas:Classification

You can mix inline schemas and Python model references in the same pipeline. Inline schemas are great for getting started; schemas.py gives you full Pydantic power when you need cross-field validators, computed properties, or shared model definitions.


Providers

pyconveyor works with any OpenAI-compatible endpoint. Just change base_url:

Provider Configuration
OpenAI provider: openai_compat
Anthropic provider: anthropic + pip install pyconveyor[anthropic]
Ollama / vLLM / LM Studio provider: openai_compat + base_url: http://localhost:11434/v1
Custom @register_provider("name") decorator
Testing provider: mock — no API calls

CLI reference

pyconveyor init <dir>                  Bootstrap a new project
pyconveyor init <dir> --interactive    Guided setup — define fields interactively
pyconveyor run <pipeline.yaml>         Run a pipeline
pyconveyor validate <pipeline>         Validate without running
pyconveyor batch <pipeline>            Batch process a JSONL file
pyconveyor benchmark <dir>             Benchmark against golden-standard cases
pyconveyor vocab review                Review pending vocabulary suggestions
pyconveyor schema                      Emit JSONSchema for editor autocomplete
pyconveyor schema infer <pipeline>     Infer schemas.py from sample output
pyconveyor visualise <pipeline>        Print Mermaid DAG diagram

Python API

from pyconveyor import PipelineRunner, BatchRunner, BenchmarkRunner, generate_report

# Single run
runner = PipelineRunner("pipeline.yaml")
result = runner.run({"paper": "..."})

result.failed                          # bool
result.steps["extract"].value          # Pydantic model or dict
result.steps["extract"].last_attempt   # AttemptLog with timing and token counts
result.summary()                       # RunSummary with aggregates

# Batch
batch_runner = BatchRunner("pipeline.yaml", max_workers=8)
for item_id, result in batch_runner.run(records):
    save(result.steps["extract"].value)

# Benchmark
bench = BenchmarkRunner("benchmarks/", pipelines=["pipeline.yaml"])
summary = bench.run()
generate_report(summary, "report.html")

Load-time validation

PipelineRunner("pipeline.yaml") validates everything before spending any tokens — model references, schema imports, expressions, step names. Errors include the YAML line number and "did you mean?" suggestions.

pyconveyor validate pipeline.yaml
# ✓ pipeline.yaml is valid

Versioning policy

The YAML pipeline format is treated as a public API subject to the same semver rules as the Python API. A breaking change to the YAML schema will increment the major version.


Documentation

Full documentation at pyconveyor.readthedocs.io


License

MIT

Metadata

Release files for pyconveyor 1.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pyconveyor 1.10.0
File Size Uploaded Provenance Actions
pyconveyor-1.10.0.tar.gz 285.4 kB With provenance Details for pyconveyor-1.10.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for pyconveyor 1.10.0
File Interpreter ABI Platform Actions
pyconveyor-1.10.0-py3-none-any.whl Python 3 none any Details for pyconveyor-1.10.0-py3-none-any.whl

Total release size: 375.0 kB

Release files / pyconveyor-1.10.0.tar.gz

Download URL pyconveyor-1.10.0.tar.gz
Size 285.4 kB
Tags Source
SHA-256 checksum
How to use checksums
8b687975e165e74cecfaa0ae0660afe25a36896dd06959a2fab7934fe72af651
BLAKE2b-256 checksum
How to use checksums
2cad4ece799f8ea9da6cb3cf2502bf1299cfa29343e05eab1492d4477974bba0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on May 14, 2026.

Transparency log

Release files / pyconveyor-1.10.0-py3-none-any.whl

Download URL pyconveyor-1.10.0-py3-none-any.whl
Size 89.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
47a7d08fb73d254bd79aa49cefc95f08557ed40c982ca3d3366606043895830c
BLAKE2b-256 checksum
How to use checksums
26bdf70ebe14e94123796d4d70f453b9e2f0afb11d4a942f9b8937c7d5536a80
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on May 14, 2026.

Transparency log

Security for pyconveyor 1.10.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for pyconveyor-1.10.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on May 14, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page