views_pipeline_core 3.1.2

A modular Python framework for end‑to‑end conflict forecasting: data ingestion, transformation, drift monitoring, model and ensemble management, evaluation, reconciliation, mapping, reporting, packaging, and artifact governance.

Acknowledgements

[!CAUTION]

🚨 ATTENTION: MANDATORY EXPLICIT CONFIGURATION (Tier 3)

As of February 2026, the pipeline enforces Explicit Intent Declaration. Legacy keys (targets, metrics) are deprecated and will trigger large warning banners.

1. Targets are MANDATORY: You must declare at least one target or the pipeline will hard-stop. 2. Be Explicit: Use task-specific prefixes (regression_ or classification_). 3. Split Metrics: Clearly separate Point metrics from Sample-based (Uncertainty) metrics.

Category	Tier 3: Best Practice (Explicit)	Legacy (Deprecated)
Targets	regression_targets, classification_targets	targets
Metrics	regression_point_metrics regression_sample_metrics classification_point_metrics classification_sample_metrics	regression_metrics classification_metrics metrics

Misspelled or unrecognized keys (e.g., regression_target without the 's') will trigger a hard stop with helpful hints.

Table of Contents

1. Conceptual Overview
2. High‑Level Architecture
3. Core Pipeline Stages
4. Managers (Orchestration Layer)
5. Modules (Functional Layer)
6. Data Layer & Querysets
7. Evaluation & Metrics
8. Reconciliation (Hierarchical Consistency)
9. Reporting & Mapping
10. CLI & Argument System
11. Configuration & Partitioning
12. Package Management
13. Logging & Monitoring
14. Development Workflow
15. Quick Start
16. FAQ

1. Conceptual Overview

The pipeline transforms raw geo‑temporal data into validated, reconciled, and documented forecasts. Key features include:

• Deterministic data preparation (queryset + transformation replay)
• Strict naming & artifact conventions
• Partition-aware evaluation (calibration/validation/forecasting)
• Multi-model ensembling & hierarchical reconciliation
• Automated HTML reporting and spatial visualization
• Reproducible configuration merging and logging
• Optional integration with Weights & Biases (WandB) and prediction store

2. High‑Level Architecture

         ┌────────────────────────────────────────┐
         │            ConfigurationManager        │
         │  (deployment + hyperparameters + meta) │
         └───────────────┬────────────────────────┘
                         │
┌────────────────────────▼─────────────────────────┐
│                 ViewsDataLoader                  │
│  Queryset → Raw Fetch → Drift Check → Update     │
│  → Transformation Replay → Partition Slice       │
└───────────────┬──────────────────────────────────┘
                │  DataFrame (month_id, entity_id)
                ▼
      ┌─────────────────────────────────────────┐
      │   ForecastingModelManager (thin façade)  │
      │   Routes to independently testable stages│
      │                                         │
      │   ┌─────────────┐  ┌──────────────────┐ │
      │   │TrainingStage│  │ EvaluationStage  │ │
      │   └─────────────┘  └──────────────────┘ │
      │   ┌───────────────┐ ┌────────────────┐  │
      │   │ForecastingStage│ │ReportingStage │  │
      │   └───────────────┘ └────────────────┘  │
      │   ┌───────────────────────┐              │
      │   │  PredictionIOManager  │              │
      │   └───────────────────────┘              │
      └────────────┬────────────────────────────┘
                   │ Predictions
                   ▼
         ┌────────────────────────┐
         │ ReconciliationModule   │
         │ (Country ↔ Priogrid)   │
         └────────────┬───────────┘
                      │ Reconciled Predictions
                      ▼
         ┌───────────────────────────┐
         │ Reporting & Mapping       │
         │ HTML, Tables, Choropleths │
         └───────────────────────────┘

Architecture note (April 2026): ForecastingModelManager has been decomposed from a 3049-LOC monolith into a ~1960-LOC thin façade that delegates to 5 independently testable stage classes. Each stage receives a frozen context object (immutable dataclass) rather than reaching into the manager's internals. ModelPathManager has been relocated from managers/ to data/ to fix dependency inversion. All existing import paths continue to work via re-export shims. See ADR-045 for the full design rationale.

3. Core Pipeline Stages

4. Managers (Orchestration Layer)

Each stage receives an immutable frozen context object and is independently unit-testable. See ADR-045 for architecture details.

5. Modules (Functional Layer)

5.1 Intermediate Modules

6. Data Layer & Querysets

• Querysets define feature/target extraction logic + transformation chains.
• Incremental updates replace raw slices (GED / ACLED) and replay transformations (UpdateViewser).
• MultiIndex structure: (month_id, entity_id) for time-spatial operations.
• Data types normalized (float64 for numeric integrity).
• Partitions defined via month ranges (train/test or forecast horizon).

7. Evaluation & Metrics

Evaluation produces:

• Step-wise metrics (per forecast horizon)
• Month-wise metrics (temporal slices)
• Time-series metrics (sequence performance trajectory)

Conflict type auto-inferred from target tokens (sb / ns / os). Files named per ADR conventions (artifact/output naming).

8. Reconciliation (Hierarchical Consistency)

Ensures priogrid sums align with authoritative country totals while preserving relative spatial pattern and zero inflation. Parallelizable across countries × time × targets. Integrated into ensembles or model forecast postprocessing.

9. Reporting & Mapping

Reports embed:

• Metrics tables
• Key–value configuration summaries
• Spatial animations (Plotly)
• Artifact provenance (timestamps, versions)

10. CLI & Argument System

Dataclass-driven (ForecastingModelArgs):

• Flags: --train, --evaluate, --forecast, --report, --sweep, --prediction_store, --monthly
• Validation prevents illegal combinations (e.g., evaluate with forecasting run type).
• Monthly shortcut auto-configures production cycle.

11. Configuration & Partitioning

ConfigurationManager merges:

1. Deployment
2. Hyperparameters
3. Meta
4. Partition dictionary
5. Runtime overrides (highest priority)

Forecast partitions dynamically adjusted by override_timestep. Validation enforces structural integrity and target specification.

12. Package Management

PackageManager:

• Validates naming (organization-prefix-*)
• Creates Poetry skeleton (Python version constraint)
• Adds dependencies (including views-pipeline-core)
• Fetches latest release (tags or GitHub API)
• Runs poetry check

13. Logging & Monitoring

• YAML-driven configuration (handlers, levels, formatters).
• Dedicated model/ensemble logging directories.
• Standard separation: main log, error log.
• WandB alerts for stage transitions, failures, reconciliation completeness.

14. Development Workflow

Refer to documentation/development_guidelines.md for coding standards and docstring_guidelines.md for formatting.

15. Quick Start

1. Run build_model_scaffold.py or build_ensemble_scaffold.py found in the views-models repository.

2. Update config_deployment.py, config_hyperparameters.py, config_queryset.py, config_meta.py.

3. Run calibration:

   python main.py --run_type calibration --train --evaluate --report
   

4. Run forecasting:

   python main.py --run_type forecasting --train --forecast --report
   

5. View artifacts: models/<name>/artifacts/

16. FAQ