stats-transformer

Table of Contents

• 1. Features
• 2. Documentation
• 3. Quickstart
  • 3.1. Installation
  • 3.2. Configuration (params.yaml)
  • 3.3. Usage
  • 3.4. Testing
• 4. Agent Skill

1. Features

stats-transformer is a Python library for data transformation, econometric modeling, and visualization. It automates feature engineering, model estimation, and reporting across time-series, panel, and cross-sectional datasets using a reproducible configuration workflow.

• Unified Empirical Workflow: Combines feature engineering, model estimation, diagnostic checks, and publication-ready charts in one pipeline.
• Declarative YAML Configuration: Controls data sources, frequency resampling, transformations, model specifications, and visual outputs via params.yaml.
• Broad Econometric Coverage: Provides unified APIs for cross-sectional (OLS, Robust OLS), panel (Fixed Effects), time-series (VAR, VECM, SVAR, Local Projections), instrumental-variables (2SLS, LP-IV), discrete-choice (Logit), and unsupervised methods (PCA, KMeans).
• Cross-Language Validation & Agent-Ready: Numerically verified against R (stats, vars), Stata (regress, logit, pca), and MATLAB (mldivide), with an embedded architectural skill for AI agents.
• Extensible Roadmap: Built for active expansion into high-frequency, non-linear, and structural macroeconomic extensions (see Roadmap).

2. Documentation

• Core Guides (docs/)

  • Overview: Documentation map, model inventory, and example index.
  • Architecture & Design: Pipeline stages, object hierarchy, and data flow.
  • Numerical Validation: Cross-language R, Stata, and MATLAB verification matrix.
  • Testing Suite: Automated unit, integration, and verification test guide.
  • Academic Citations: Literature sources, paper datasets, and reference software.
  • Roadmap & Extensions: Planned model expansions and frequentist VAR milestones.
  • File Structure: Cookiecutter-based research folder layout.

• Interactive Notebooks (notebooks/)

  • Overall Pipeline: End-to-end data ingestion, transformation, and estimation.
  • Regression & Panel: OLS, robust covariance, panel, and 2SLS examples.
  • Time Series & Structural VAR: VAR, SVAR, local projection, and decomposition APIs.
  • Modular Chart Components: Interactive visualization components and publication plotting.

3. Quickstart

3.1. Installation

To use it in your project via PyPI:

pip install stats-transformer

Or add it with uv:

uv add stats-transformer

For local development from this repository:

uv sync

3.2. Configuration (params.yaml)

Define your data sources, pipeline parameters, and model specifications in a params.yaml file:

data:
  featurization:
    entity_column: country
  datasets:
    - name: macro_data
      path: data/raw/macro_indicators.csv
      frequency: Q

model:
  model_type: panel_ols
  target_variable: gdp_growth
  independent_variables:
    - interest_rate
    - inflation

visualization:
  output_dir: reports/visualizations

3.3. Usage

Load a packaged example dataset:

from stats_transformer.data import list_examples, load_example

print(list_examples())
df = load_example("macrodb_gdp_inflation")

You can execute the pipeline via the command line using the Pipeline orchestrator:

# Run the full end-to-end pipeline
uv run python -m stats_transformer.pipeline --config params.yaml

Or you can interact with the API programmatically:

from stats_transformer import Pipeline

# Initialize the pipeline with your configuration
pipeline = Pipeline(params_path="params.yaml")

# Run specific stages sequentially
merged_data = pipeline.run(stage="resample")
transformed_data = pipeline.run(stage="features")
model_results = pipeline.run(stage="regression")

# Generate and save visualizations
pipeline.run(stage="visualization")

3.4. Testing

Verify the installation and library integrity by running the test suite:

uv run python -m pytest -q

For more details on test coverage, see the Testing Suite.

4. Agent Skill

Source checkouts of this repository include an optional stats-transformer-architecture agent skill for AI coding tools. It gives agents a compact map of the library architecture, pipeline stages, model contracts, and feature-engineering vocabulary.

The canonical skill source lives at .agents/skills/stats-transformer-architecture/. From the repository root, use the local terminal to run scripts/install-agent-skill.sh, which installs or refreshes tool-specific copies for Claude Code, OpenAI Codex, or Kilo Code:

./scripts/install-agent-skill.sh all

You can also target one tool at a time:

./scripts/install-agent-skill.sh claude
./scripts/install-agent-skill.sh codex
./scripts/install-agent-skill.sh kilo