deep-inference 0.2.0

Production-grade econometrics with neural networks

deep-inference

Deep Learning for Individual Heterogeneity

References:

• Farrell, Liang, Misra (2021) "Deep Neural Networks for Estimation and Inference" Econometrica
• Farrell, Liang, Misra (2025) "Deep Learning for Individual Heterogeneity"

---

deep-inference is a Python package for enriching structural economic models with deep learning. It implements the framework developed by Farrell, Liang, and Misra (2021, 2025) to recover rich, non-linear parameter heterogeneity ($\theta(X)$) while maintaining the interpretability and validity of structural economics.

Standard deep learning minimizes prediction error, which leads to biased parameter estimates ("The Inference Trap"). This package implements Influence Function-based Debiasing to provide valid confidence intervals and p-values for economic targets.

Why deep-inference?

Economic structure and Machine Learning are complements, not substitutes.

• Deep Learning provides the capacity to learn complex, high-dimensional heterogeneity.
• Structural Models provide the constraints necessary for causal interpretation and counterfactuals.

deep-inference enforces the structural loss (e.g., Tobit likelihood) on the output of a neural network, ensuring that the estimated parameters $\alpha(X), \beta(X)$ respect the economic theory.

Installation

pip install deep-inference

Quickstart

Estimate a linear demand model where Price Elasticity $\beta$ varies non-linearly with customer characteristics $X$.

import numpy as np
from deep_inference import structural_dml

# 1. Generate synthetic data (Linear Demand with Heterogeneity)
np.random.seed(42)
n = 2000
X = np.random.randn(n, 10)  # Customer characteristics
T = np.random.randn(n)       # Treatment (e.g., price)
beta_true = np.cos(np.pi * X[:, 0]) * (X[:, 1] > 0) + 0.5 * X[:, 2]
Y = X[:, 0] + beta_true * T + np.random.randn(n)
mu_true = beta_true.mean()

# 2. Run Inference
# The package trains the structural network, computes the influence function,
# and aggregates results via cross-fitting.
result = structural_dml(
    Y=Y, T=T, X=X,
    family='linear',
    hidden_dims=[128, 64, 32],
    epochs=50,
    lr=0.01,
    n_folds=50,
)

print(f"True mu* = {mu_true:.4f}")
print(result.summary())

New inference() API

The new inference() API provides additional flexibility:

• Flexible targets: Average Marginal Effect (AME), custom target functions
• Randomization mode: For RCTs, compute Λ directly instead of estimating
• Regime auto-detection: Automatically chooses optimal Lambda strategy

from deep_inference import inference
from deep_inference.lambda_.compute import Normal

# Average Marginal Effect (probability scale, not log-odds)
result = inference(Y, T, X, model='logit', target='ame', t_tilde=0.0)

# Custom target with autodiff Jacobian
import torch
def my_target(x, theta, t_tilde):
    return torch.sigmoid(theta[0] + theta[1] * t_tilde)

result = inference(Y, T, X, model='logit', target_fn=my_target)

# Randomized experiment (Regime A)
result = inference(Y, T, X, model='logit', target='beta',
                   is_randomized=True, treatment_dist=Normal(0, 1))

Three Estimation Regimes

Regime	Condition	Lambda Method	Cross-Fitting
A	RCT with known treatment distribution	Compute via MC	2-way
B	Linear structural model	Closed-form analytic	2-way
C	Observational, nonlinear	Neural network estimation	3-way

Supported Structural Families

The package abstracts the math of influence functions. You simply select the family that matches your outcome variable.

Family	Model Structure	Use Case
Linear	$Y = \alpha(X) + \beta(X)T + \varepsilon$	Wages, Test Scores, Consumption
Logit	$P(Y=1) = \sigma(\alpha(X) + \beta(X)T)$	Binary Choice, Market Entry
Poisson	$Y \sim \text{Pois}(\exp(\alpha(X) + \beta(X)T))$	Patent Counts, Doctor Visits
Gamma	$Y \sim \text{Gamma}(k, \exp(\alpha(X) + \beta(X)T))$	Healthcare Costs, Insurance Claims
Gumbel	$Y \sim \text{Gumbel}(\alpha(X) + \beta(X)T, s)$	Extreme Value Analysis
Tobit	$Y = \max(0, \alpha(X) + \beta(X)T + \sigma\varepsilon)$	Labor Supply, Censored Demand
NegBin	$Y \sim \text{NegBin}(\exp(\alpha(X) + \beta(X)T), r)$	Count Data with Overdispersion
Weibull	$Y \sim \text{Weibull}(k, \exp(\alpha(X) + \beta(X)T))$	Duration Analysis, Survival
Multinomial Logit	$P(Y=j) = \text{softmax}(\alpha_j(X) + X'_j \beta(X))$	Transportation, Brand Choice

Note: For complex models like Tobit, deep-inference automatically handles the joint estimation of structural variance $\sigma(X)$ required for consistent inference.

Methodological Details

1. The Enriched Model

We replace fixed parameters $\theta$ with neural networks $\theta(X)$:

\hat{\theta}(\cdot) = \arg \min_{\theta \in \mathcal{F}_{DNN}} \sum \ell(y_i, t_i, \theta(x_i))

2. The Influence Function Correction

Naive averaging of $\hat{\theta}(X)$ yields biased inference. We construct a Neyman-Orthogonal score $\psi$ using the Influence Function:

\psi(z) = H(\hat{\theta}) + \nabla_\theta H \cdot \Lambda(x)^{-1} \cdot \nabla_\theta \ell(z, \hat{\theta})

Where $\Lambda(x) = \mathbb{E}[\nabla^2 \ell \mid X=x]$ is the conditional Hessian.

• Automatic Differentiation: deep-inference uses PyTorch Autograd to compute exact Jacobians and Hessians for any model family.
• Stability: Includes Tikhonov regularization for inverting Hessians in non-linear models (e.g., Logit/Tobit).

Validation

Comprehensive eval suite validates every mathematical component. Full results →

Eval	Component	Result
01	Parameter Recovery	12/12 families PASS
02	Autodiff Accuracy	31/31 PASS
03	Lambda Estimation	9/9 PASS
04	Target Jacobian	92/92 PASS
05	Influence Functions	Coverage 88%
06	Frequentist Coverage	PASS
07	End-to-End	7/7 PASS
09	Multinomial Logit	98% coverage PASS

Citation

@article{farrell2021deep,
  title={Deep Neural Networks for Estimation and Inference},
  author={Farrell, Max H. and Liang, Tengyuan and Misra, Sanjog},
  journal={Econometrica},
  volume={89},
  number={1},
  pages={181--213},
  year={2021}
}

@article{farrell2025heterogeneity,
  title={Deep Learning for Individual Heterogeneity},
  author={Farrell, Max H. and Liang, Tengyuan and Misra, Sanjog},
  journal={Working Paper},
  year={2025}
}

License

MIT