PyCharter

Work management

In this repository, backlog means the Optophi — Delivery Project. The PyCharter backlog is its repository-filtered view, using Record kind: Work. Reference records are historical context. GitHub Issues hold scope, acceptance criteria, dependencies and evidence; Project fields hold current priority and delivery status. Backlog is a status within that same Project, not a separate task list.

Do not maintain a parallel Markdown or Linear backlog. Historical sources and legacy-ID mappings are retained through the migration archive; they are provenance only. When adding work to the Project, set Record kind: Work. Retained proposals, program boards and status tables are design references or dated history; their presence does not authorize implementation. Search existing Issues and follow migration mappings to the canonical Issue before creating work. Link advancing PRs and keep local qualification, merge, release and deployment evidence separate. Technical specifications remain in their owning documents.

Python users · Coding agents

These source docs include develop features. See the Python guide for the published-release boundary and a pinned offline quickstart.

Define good data once. Check it wherever it flows.

PyCharter turns data requirements into executable, versioned contracts. Use the same rules to validate application records, check a batch, guard a pipeline step, or verify an agent's output. When data fails, get an explanation you can act on.

Start with one Python call. Grow toward a reliable data feed with reviewed pipelines, inspectable delivery and a clear recovery path.

For Python developers checking records before an application consumes them, the smallest useful workflow is a local validator or file check. The first-result walkthrough uses one order contract to accept data, explain a rejection and check the corrected record.

Get started · Build a data feed · The complete product journey · Documentation

Validate a record in 30 seconds

python -m pip install pycharter

The source core requires Python 3.11+ and Pydantic >=2.11.0,<3.0. When upgrading, update any older explicit Pydantic pin to that supported range.

from pycharter import Validator

validator = Validator.from_dict(
    {
        "type": "object",
        "version": "1.0.0",
        "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
        },
        "required": ["name", "age"],
    }
)

result = validator.validate({"name": "Alice", "age": 30})
print(result.is_valid)  # True

result = validator.validate({"name": "Bob", "age": "N/A"})
print(result.errors[0])  # Column 'age': expected integer, found string 'N/A'

This runs locally without a server, database, account or API key. By default, compatible values such as numeric strings are coerced before validation; use strict_types=True when constructing the validator to require strict types. Load a saved contract with Validator.from_file("contract.yaml") when you want the same requirements in another application or tool.

Prefer the shell? Create a contract document and a tiny local input file. The Python factory above accepts a raw JSON Schema. For new contract files, use the native v2 envelope shown in the first-result guide. Supported v1 files remain loadable:

cat > contract.yaml <<'YAML'
api_version: pycharter.io/v2
kind: DataContract
metadata:
  name: person_record
  version: "1.0.0"
spec:
  json_schema:
    type: object
    version: "1.0.0"
    properties:
      name: {type: string}
      age: {type: integer}
    required: [name, age]
YAML
printf '{"name":"Alice","age":30}\n' > data.jsonl
pycharter validate contract.yaml data.jsonl

This reports one valid record and exits 0. Run the check again to repeat it. To inspect a deliberate failure:

printf '{"name":"Bob","age":"N/A"}\n' > data.jsonl
pycharter validate contract.yaml data.jsonl

This exits 1 and identifies row 0 and column age. Restore the valid input above and rerun the same command. This basic path needs no database or running service.

The first-result guide provides complete commands, expected counts and exit codes for Python and the file CLI. Its checked-in inputs include the deliberate failure and its correction; the guide also creates these files without a clone. The exercise uses the published 0.0.67 package. Repository examples are separate from installed package assets. The README examples also run in the source wheel's minimum/current dependency gates.

Two ways to start

You can keep contracts in your repository. Add a shared registry, services or an external scheduler when your workflow needs them.

What changes when the contract is executable

Requirements travel with the data. A contract combines structure, permitted conversions, validation rules and optional quality or business metadata. People can review the file; Python can enforce it.

Failures become specific. Find the record, field and failed requirement. Choose a supported fail or quarantine policy at the enforcement boundary. An inferred contract is a draft to review, not proof that the sample captured every valid case.

Pipelines and requirements stay connected. Declare extraction, transformation and loading in YAML, and attach a contract to the steps that must enforce it. A contract stored beside a pipeline does not enforce itself. Try the local pipeline.

Agents work against checks you control. Give an agent the relevant schema and context, validate its candidate, and review changes to the requirements. The validator supplies the verdict; the agent supplies a proposed answer or repair. Contract conformance checks declared requirements, not factual truth.

Operation has a clear destination. The complete experience connects first validation, verified delivery, scheduling, independent health and recovery. The product journey describes that end state; capability status distinguishes available features, bounded references and work still being completed.

Use the interface that fits your work

The product direction is one coherent workflow across these interfaces. Use the capability guide for the current scope of each surface; a configured MCP endpoint or UI does not itself provide scheduling.

From an application check to an operated data feed

1. Describe acceptable data. Start from a sample, existing schema or contract. Review meaning, required fields and failure policy.
2. Verify a bounded example. Inspect accepted and rejected data before introducing live source credentials or destination changes.
3. Deliver and inspect. Enforce the contract at the selected boundary and inspect the actual output and execution result.
4. Operate and recover. Add scheduling, durable state and independent monitoring in a qualified deployment profile.

The bounded FMP reference demonstrates the operational direction: exact coverage checks, PostgreSQL publication receipts, safe reruns and an independent monitor. It currently requires a reviewed source checkout; the public 0.0.67 package cannot run that reference. Its fixture uses synthetic data. Live access, release packaging and recurring production operation have separate qualification gates.

For continuous data, the streaming engine applies contracts record by record with rollout modes and rejected-record routing. Streaming remains experimental; delivery guarantees depend on the selected broker, sink and mode.

Add capabilities as you need them

python -m pip install pycharter                     # Local validation
python -m pip install "pycharter[pipeline,postgres]" # Additional connectors and Postgres drivers
python -m pip install "pycharter[server]"            # API, UI host and worker dependencies
python -m pip install "pycharter[ai]"                # MCP and agent integration dependencies

These install dependencies. Connection setup, credentials, service operation and scheduling are explicit choices. See Installation for streaming, semantic and orchestrator extras.

Minimal example (contract + pipeline + check)

This example is qualified against the source wheel. For public 0.0.67, follow the file-pipeline installation note to install sqlalchemy[asyncio] before running it.

# 1) Define a contract
cat > contract.yaml <<'YAML'
api_version: pycharter.io/v2
kind: DataContract
metadata: { name: demo_contract, version: "1.0.0" }
spec:
  json_schema:
    type: object
    version: "1.0.0"
    required: [email, amount]
    properties:
      email:  { type: string, format: email }
      amount: { type: number, minimum: 0 }
YAML

# 2) Prepare a tiny input file
printf '{"email":"a@x.io","amount":10}\n{"email":"bad","amount":-1}\n' > input.jsonl

# 3) A file->file pipeline that copies the data (no network)
mkdir -p pipeline && cat > pipeline/pipeline.yaml <<'YAML'
api_version: pycharter.io/v2
kind: Pipeline
metadata: { name: copy_demo, version: "1.0.0" }
spec:
  variables: { IN: input.jsonl, OUT: output.jsonl }
  steps:
    - id: read
      type: extract
      source: { type: file, path: "${IN}", format: jsonl }
    - id: write
      type: load
      input: read
      target: { type: file, path: "${OUT}", format: jsonl, mode: overwrite }
YAML

# 4) Run the pipeline, then validate its output against the contract
pycharter pipeline run ./pipeline --watch
pycharter validate contract.yaml output.jsonl

The pipeline copies both rows and exits 0. The separate validation command intentionally exits 1: row 1 has an invalid email and a negative amount. This example checks the copied output; it does not attach enforcement to the pipeline steps. Rerunning overwrites the output with the same two input rows.

To enforce the contract during delivery, continue with the local data-feed guide.

Keep your requirements portable

Use YAML/JSON contracts, version them with your code, and exchange the supported ODCS subset with other tools. Shared vocabularies and field bindings add business meaning when you need it. Start with the smallest useful contract and introduce those capabilities progressively.

• Contract formats and versioning
• ODCS import/export and supported semantics
• Vocabulary and field meaning
• Where PyCharter fits in your stack

Documentation and contribution

Read the documentation for task guides, API references and operating instructions. The cookbook contains focused recipes for applications, files, streams and quality checks.

For development, follow CONTRIBUTING.md, ARCHITECTURE.md and AGENTS.md. Read CHANGELOG.md for release evidence, source-only Unreleased changes and upgrade guidance. Report security issues privately under SECURITY.md. Community expectations are in CODE_OF_CONDUCT.md.

Licensed under the Apache License, Version 2.0. See LICENSE.