stats-transformer

Table of Contents

• 1. Features
• 2. Documentation
• 3. Quickstart
  • 3.1. Installation
  • 3.2. Usage
    • 3.2.1. With params.yaml
    • 3.2.2. With the API directly
  • 3.3. Testing
• 4. Agent Skill

1. Overview

stats-transformer is a Python library for data transformation, econometric modeling, and visualization. It automates feature engineering, model estimation, and reporting across time-series, panel, and cross-sectional datasets using a reproducible configuration workflow.

• Reproducible Workflows: Research decisions from data ingestion to publication output are driven by a single config (params.yaml), in four stages; each run lands in a timestamped run folder with a provenance manifest.json (config and input checksums, git SHA, stages run, written files)
  • Data stage: ingest, resample, merge
  • Features stage: transform into analysis variables
  • Model stage: estimate with diagnostics and a JSON summary
  • Reporting stage: charts and publication tables
• Broad Econometric Coverage: Covers cross-sectional (OLS, robust OLS), panel (fixed effects), time-series (VAR, VECM, SVAR, local projections), IV (2SLS), discrete choice (Logit), and unsupervised methods (PCA, KMeans).
  • Validated via legacy software: Model estimation and transformation validated against MATLAB, R, and Stata

2. Documentation

• Core Guides (docs/)

  • Overview: Documentation map, model domain summary, and quickstart links.
  • Architecture & Design: Pipeline stages, object hierarchy, and repository file structure.
  • Implemented Models: Complete catalog of all 32 implemented model classes and utilities.
  • Data Guide: Data directory organization and packaged example datasets.
  • Software Benchmarks: Cross-language R, Stata, and MATLAB verification matrix.
  • Testing Suite: Automated unit, integration, and verification test guide.
  • Academic Citations: Literature sources, paper datasets, and reference software.
  • Future Roadmap: Planned model expansions and complexity triage.

• Interactive Notebooks (notebooks/)

  • Overall Pipeline: End-to-end data ingestion, transformation, and estimation.
  • Regression & Panel: OLS, robust covariance, panel, and 2SLS examples.
  • Time Series & Structural VAR: VAR, SVAR, local projection, and decomposition APIs.
  • Modular Chart Components: Interactive visualization components and publication plotting.

3. Quickstart

3.1. Installation

To use it in your project via PyPI:

pip install stats-transformer

Or add it with uv:

uv add stats-transformer

To run the notebooks under notebooks/, install the notebooks extra:

pip install "stats-transformer[notebooks]"

For local development from this repository:

uv sync

3.2. Usage

3.2.1. With params.yaml

Define your data sources, pipeline parameters, and model specifications in a params.yaml file:

data:
  raw_data_file: data/examples/regression/grunfeld.csv
  featurization:
    entity_column: country
    date_column: date
    column_mapping:
      year: date
      firm: country

model:
  model_type: panel_ols
  target_variable: invest
  independent_variables:
    - value
    - capital

reporting:
  output_dir: reports/visualizations

Execute the full pipeline via the command line using the Pipeline orchestrator:

uv run python -m stats_transformer.pipeline --config params.yaml

Or drive the same configuration programmatically:

from stats_transformer import Pipeline

pipeline = Pipeline(params_path="params.yaml")
merged_data = pipeline.run(stage="data")
transformed_data = pipeline.run(stage="features")
model_results = pipeline.run(stage="model")
pipeline.run(stage="reporting")

3.2.2. With the API directly

Skip the YAML file and instantiate model classes directly. Start from a packaged example dataset:

from stats_transformer.data import list_examples, load_example

print(list_examples())
df = load_example("macrodb_gdp_inflation")

Then fit any model class on the data:

from stats_transformer.models.regression.robust_ols import RobustOLSModel
from stats_transformer.reporting.models.regression_viz import RegressionVisualizer

model = RobustOLSModel(
    target="gdp",
    independent_variables=["inflation"],
    cov_type="HC3",
)
model.fit(df)
print(model.get_summary())

viz = RegressionVisualizer(output_dir="reports/visualizations")
paths = viz.visualize_from_json(model_summary=model.get_model_metadata())

All 32 model classes accept direct instantiation like this; see Implemented Models for each class's arguments. End-to-end examples live in the notebooks.

3.3. Testing

Verify the installation and library integrity by running the test suite:

uv run python -m pytest -q

For more details on test coverage, see the Testing Suite.

4. Agent Skill

Source checkouts of this repository include an optional stats-transformer-architecture agent skill for AI coding tools. It gives agents a compact map of the library architecture, pipeline stages, model contracts, and feature-engineering vocabulary.

The canonical skill source lives at .agents/skills/stats-transformer-architecture/. From the repository root, use the local terminal to run scripts/install-agent-skill.sh, which installs or refreshes tool-specific copies for Claude Code, OpenAI Codex, or Kilo Code:

./scripts/install-agent-skill.sh all

You can also target one tool at a time:

./scripts/install-agent-skill.sh claude
./scripts/install-agent-skill.sh codex
./scripts/install-agent-skill.sh kilo