views-postprocessing

The post-forecast delivery layer for the VIEWS (Violence Early-Warning System) pipeline. It takes finished VIEWS forecasts, enriches them with geographic metadata, guards their integrity, and delivers them to a partner store.

Two partner deliveries run on the same partner-neutral machinery in contract/: the UN FAO path (views_postprocessing/unfao/), serving FAO-FSFC since 2026-07-27, and CRAF'd (views_postprocessing/crafd/), added 2026-08-03 with its upload interlock still closed.

New here? Read docs/architecture/role_and_seams.md first. It explains what this repo is, how it relates to pipeline-core / faoapi / datafactory, and where its internal seams are. This README is install + quickstart only.

What this is (and isn't)

• It is a concrete pipeline-core postprocessor: UNFAOPostProcessorManager subclasses pipeline-core's PostprocessorManager and fills the post-forecast lifecycle (read → transform → validate → save) for the FAO partner.
• It enriches and delivers — it joins GAUL administrative metadata onto predictions (via a precomputed lookup, ADR-011) and enforces input-integrity invariants before upload.
• It is not a spatial-mapping library (the old runtime spatial mapper was removed — see ADR-011 / C-39) and not a statistical post-processor. Draw-collapse (MAP/HDI) happens downstream in views-faoapi; reconciliation lives in views_frames_reconcile. See the orientation doc.

Installation

# With Poetry (recommended)
git clone https://github.com/views-platform/views-postprocessing.git
cd views-postprocessing
poetry install

# Or with pip
pip install views-postprocessing

Requires Python 3.11–3.14.

Dependencies

The UN FAO delivery

from views_pipeline_core.managers.postprocessor import PostprocessorPathManager
from views_postprocessing.unfao.managers.unfao import UNFAOPostProcessorManager

path_manager = PostprocessorPathManager("un_fao")
manager = UNFAOPostProcessorManager(model_path=path_manager)
manager.execute()   # read → transform → validate → save

In practice the manager is constructed and run by views-models (postprocessors/un_fao/main.py), not invoked directly.

Pipeline stages

The pandas metadata-join and history-clip stages were retired with the legacy delivery path in #149; their rules survive as called invariants under delivery/. See the manager README for what moved where.

If a delivered value turns out to be wrong

docs/operations/correction_procedure.md — how to establish which deliveries are affected, confirm the fault offline, and supersede on the wire. The contract has no retraction primitive; a correction is a new complete run, manifest last.

Output schema (geographic metadata columns)

These 9 columns are the delivered geography contract, declared in contract/gaul_schema.py. The order below is normative (ADR-013 §5.1) and is byte-pinned by the §10 golden fixture — a reader that reorders them reads the wrong column. tests/test_doc_accuracy.py checks this table against the declaration.

(Corrected 2026-08-03: this table had admin1_gaul0_* before admin1_gaul1_* — the reverse of the normative order — and typed the four *_code columns int. They are float64 on the wire, always, by the §5.1 ruling: the codes are nullable and arrow has no nullable int in this contract. Both errors survived because nothing compared the table to the declaration.)

Package structure

views-postprocessing/
├── pyproject.toml
├── README.md
├── docs/
│   ├── architecture/role_and_seams.md   # READ FIRST — role + seams
│   ├── ADRs/                             # decisions + rationale
│   └── CICs/                             # class-level contracts
└── views_postprocessing/
    ├── delivery/                 # WHAT MAKES A DELIVERY VALID — representation-free
    │   ├── coverage.py             # region cell-count + excluded-cell guards
    │   ├── draws.py                # the §6 no-collapse gate
    │   ├── parity.py               # sidecar covers exactly the forecast's cells
    │   ├── observed_range.py       # fabricated-month decision
    │   └── provenance.py           # structured upload provenance
    ├── contract/                 # HOW A DELIVERY IS BUILT — partner-neutral
    │   ├── wire/                    # the ADR-013 contract (header, shard, sidecar,
    │   │                            #   run_manifest, sink, source_selection, naming)
    │   ├── frames.py                # PredictionFrame / TargetFrame constructors
    │   ├── frame_extraction.py      # THE representation seam (frame → primitives)
    │   ├── track_a_source.py        # Hop-A archive → frame
    │   ├── historical.py            # the historical artifact, built pandas-free
    │   ├── gaul_lookup.py           # the GAUL asset: path, version, one read
    │   ├── gaul_schema.py           # the 9-column contract, declared as data
    │   ├── source_metadata.py       # producer (datafactory) facts
    │   ├── store_metadata.py        # prediction-store facts
    │   └── launch_config.py         # the delivery mode the launcher must declare
    ├── unfao/                    # WHO A DELIVERY IS FOR — the FAO-specific code
    │   ├── product.py               # targets, consumer name, S_MIN, upload interlock
    │   ├── appwrite_env.py          # the declared store coordinates
    │   └── managers/unfao.py        # UNFAOPostProcessorManager
    ├── crafd/                    # WHO A DELIVERY IS FOR — the CRAF'd-specific code
    │   ├── product.py               # same three files, same shape (register C-33 on
    │   ├── appwrite_env.py          #   why the manager is a copy, and what would
    │   └── managers/crafd.py        #   make it time to stop copying)
    └── data/gaul_lookup.parquet  # the precomputed GAUL lookup (ADR-011)

Dependencies point one way only: <partner>/ → contract/ → delivery/. Nothing in contract/ may import a partner package — that is what lets a new partner reuse the machinery without inheriting another partner's product, and it is enforced by tests/test_clone_readiness.py, not by convention. The partner list lives in one place (tests/conftest.py) and is itself checked against the filesystem, so a package added without being declared fails rather than passing quietly. See docs/CLONING.md.

Configuration

Each delivery reads Appwrite connection settings from the environment. The required names are declared per partner — unfao/appwrite_env.py, crafd/appwrite_env.py — and validated fail-loud before any store is constructed: a missing or empty variable raises, naming every one that is absent, rather than half-configuring a client.

The names are below; the values are not. Coordinates live in the Appwrite Seam Contract's registry, which this repo references by pinned URL and never copies (þing-01 sáttmál S6 — copies were the platform's original failure). The launcher supplies the values; the API key is an operator slot.

# Appwrite connection
APPWRITE_ENDPOINT=...
APPWRITE_DATASTORE_PROJECT_ID=...
APPWRITE_DATASTORE_API_KEY=...        # operator-issued secret

# Production-forecasts store (input — shared by every partner)
APPWRITE_PROD_FORECASTS_BUCKET_ID=...
APPWRITE_PROD_FORECASTS_BUCKET_NAME=...
APPWRITE_PROD_FORECASTS_COLLECTION_ID=...
APPWRITE_PROD_FORECASTS_COLLECTION_NAME=...

# UN FAO store (output)
APPWRITE_UNFAO_BUCKET_ID=...
APPWRITE_UNFAO_BUCKET_NAME=...
APPWRITE_UNFAO_COLLECTION_ID=...
APPWRITE_UNFAO_COLLECTION_NAME=...

# CRAF'd store (output)
APPWRITE_CRAFD_BUCKET_ID=...
APPWRITE_CRAFD_BUCKET_NAME=...
APPWRITE_CRAFD_COLLECTION_ID=...
APPWRITE_CRAFD_COLLECTION_NAME=...

# Metadata database (shared)
APPWRITE_METADATA_DATABASE_ID=...
APPWRITE_METADATA_DATABASE_NAME=...

(Corrected 2026-08-03: four production-forecasts coordinate values were written out above, two lines below the sentence saying they never are. The value-copy guard scanned only .py; it now scans markdown too.)

Documentation

Contributing

1. Branch off development.
2. Make the change; keep ruff and the test suite green (ruff check . && PYTHONPATH=. pytest -q).
3. Open a PR into development.

Contributor protocols (incl. the conventions for AI agents) are under docs/contributor_protocols/.

License

MIT — part of the VIEWS platform developed at the Peace Research Institute Oslo (PRIO). See LICENSE.

Related packages