pyoptima 0.0.3

Universal optimization framework - solve LP, QP, MIP, knapsack, TSP, scheduling, and more with a simple API

PyOptima

Portfolio Optimization Engine - Registry-based optimization framework with ETL integration

PyOptima is a Python package for portfolio optimization and other optimization problems. It provides a registry-based method system, supports advanced constraints, and integrates seamlessly with ETL pipelines.

Features

• 🚀 Method-Based Configuration - Define optimizations by method name (e.g., "min_volatility", "max_sharpe")
• 🔧 Multiple Solvers - Support for IPOPT (quadratic/nonlinear), HiGHS (linear/MIP), CBC, GLPK, and Gurobi via Pyomo
• 📋 Advanced Constraints - Sector caps, cardinality, turnover limits, per-asset bounds, minimum position sizes
• 🎯 ETL Integration - Direct compatibility with statfyi-optimization-api ETL pipeline
• 🔄 Data Format Flexibility - Automatic detection and conversion of various data formats (nested dict, flat dict, numpy, pandas)
• 🏗️ Registry System - Easy to add new optimization methods
• 📊 21 Portfolio Methods - Comprehensive portfolio optimization methods
• 🌐 API Server - FastAPI-based REST API for optimization requests
• 💻 Web UI - Next.js-based interactive optimization interface

Installation

Core Library

pip install pyoptima

With Optional Dependencies

# API server
pip install pyoptima[api]

# UI (Next.js frontend)
pip install pyoptima[ui]

# All optional dependencies
pip install pyoptima[api,ui]

Solvers

PyOptima uses Pyomo, which supports multiple solvers:

• IPOPT (default) - Quadratic/nonlinear optimization (required for portfolio optimization)
• HiGHS - Linear and mixed-integer programming (included via pip)
• CBC - Linear/integer programming
• GLPK - Linear programming
• Gurobi - Commercial solver (requires license)

Installation:

• HiGHS: Automatically installed with PyOptima (pip install pyoptima)
• IPOPT: Requires IPOPT C++ libraries first, then pip install pyoptima (includes cyipopt)
  • Install IPOPT libraries: conda install -c conda-forge ipopt (recommended)
  • Or use system packages: sudo apt-get install coinor-libipopt-dev (Ubuntu/Debian)
  • PyOptima automatically uses the ipopt_v2 interface via cyipopt for better performance
• See Solver Installation Guide for details

Quick Start

1. Basic Portfolio Optimization

Create a configuration file (portfolio.json):

{
  "meta": {
    "job_id": "portfolio-001",
    "solver": "ipopt"
  },
  "method": "min_volatility",
  "parameters": {
    "weight_bounds": [0, 1]
  },
  "data": {
    "expected_returns": [0.12, 0.11, 0.15],
    "covariance_matrix": [
      [0.04, 0.01, 0.02],
      [0.01, 0.05, 0.01],
      [0.02, 0.01, 0.06]
    ]
  }
}

2. Run Optimization

Using CLI

pyoptima optimize portfolio.json

Using Python API

from pyoptima import OptimizationEngine

engine = OptimizationEngine()
result = engine.optimize_from_file("portfolio.json")

print(f"Weights: {result['weights']}")
print(f"Portfolio Return: {result['portfolio_return']:.4f}")
print(f"Portfolio Volatility: {result['portfolio_volatility']:.4f}")

3. With Constraints

{
  "meta": {
    "job_id": "portfolio-002",
    "solver": "ipopt"
  },
  "method": "max_sharpe",
  "parameters": {
    "risk_free_rate": 0.02
  },
  "constraints": {
    "sector_caps": {"Technology": 0.40},
    "asset_sectors": {
      "AAPL": "Technology",
      "MSFT": "Technology",
      "GOOGL": "Technology"
    },
    "max_positions": 10,
    "min_position_size": 0.02
  },
  "data": {
    "expected_returns": [0.12, 0.11, 0.15],
    "covariance_matrix": [[0.04, 0.01, 0.02], [0.01, 0.05, 0.01], [0.02, 0.01, 0.06]]
  }
}

4. Programmatic Usage

from pyoptima import OptimizationEngine
from pyoptima.models.config import ConstraintsConfig, Meta, OptimizationConfig
import numpy as np
import pandas as pd

# Create engine
engine = OptimizationEngine()

# Create optimizer from config
config = OptimizationConfig(
    meta=Meta(job_id="portfolio-001", solver="ipopt"),
    method="min_volatility",
    constraints=ConstraintsConfig(
        sector_caps={"Technology": 0.40},
        asset_sectors={"AAPL": "Technology", "MSFT": "Technology"},
        max_positions=10
    )
)

optimizer = engine.create_from_config(config)

# Optimize with data
expected_returns = pd.Series([0.12, 0.11, 0.15], index=["AAPL", "MSFT", "GOOGL"])
covariance_matrix = pd.DataFrame(
    [[0.04, 0.01, 0.02], [0.01, 0.05, 0.01], [0.02, 0.01, 0.06]],
    index=["AAPL", "MSFT", "GOOGL"],
    columns=["AAPL", "MSFT", "GOOGL"]
)

result = optimizer.optimize(
    expected_returns=expected_returns,
    covariance_matrix=covariance_matrix
)

print(f"Weights: {result['weights']}")
print(f"Portfolio Return: {result['portfolio_return']:.4f}")
print(f"Portfolio Volatility: {result['portfolio_volatility']:.4f}")

ETL Integration

PyOptima integrates directly with the statfyi-optimization-api ETL pipeline. It automatically handles the consolidated inputs format from opt_portfolio_optimization_inputs:

from pyoptima.integration.etl import optimize_from_etl_inputs
from pyoptima.models.config import ConstraintsConfig

# Consolidated inputs from database (ETL format)
consolidated_inputs = {
    "watchlist_type": "growth",
    "week_start_date": "2025-01-06",
    "symbols": ["AAPL", "MSFT", "GOOGL"],
    "covariance_matrix": {
        "matrix": [[0.04, 0.01, 0.02], [0.01, 0.05, 0.01], [0.02, 0.01, 0.06]],
        "symbols": ["AAPL", "MSFT", "GOOGL"]
    },
    "expected_returns": {"AAPL": 0.12, "MSFT": 0.11, "GOOGL": 0.15},
    "covariance_method": "sample_cov",
    "expected_returns_method": "mean_historical",
}

# Optional constraints
constraints = ConstraintsConfig(
    sector_caps={"Technology": 0.40},
    asset_sectors={"AAPL": "Technology", "MSFT": "Technology", "GOOGL": "Technology"},
    max_positions=10,
    min_position_size=0.02
)

# Optimize - returns ETL-compatible format
result = optimize_from_etl_inputs(
    consolidated_inputs,
    method="min_volatility",
    constraints=constraints,
    solver="ipopt"
)

# Result includes weights, metrics, and metadata
print(result["weights"])
print(result["portfolio_return"])
print(result["optimization_metadata"])

Key Features:

• Automatic format detection and conversion
• Symbol alignment between covariance and expected returns
• ETL-compatible result format
• Support for all constraint types

See ETL Integration Guide for complete details.

Constraints

PyOptima supports advanced portfolio constraints beyond basic weight bounds:

Available Constraints

• Sector Caps - Limit total exposure to specific sectors
• Sector Minimums - Enforce minimum exposure to sectors
• Cardinality - Maximum number of non-zero positions
• Turnover Limits - Control trading from current portfolio state
• Per-Asset Bounds - Individual min/max bounds per asset
• Minimum Position Size - Conditional holding requirements

Example

from pyoptima.models.config import ConstraintsConfig

constraints = ConstraintsConfig(
    # Sector constraints
    sector_caps={"Technology": 0.40, "Financials": 0.30},
    asset_sectors={
        "AAPL": "Technology",
        "MSFT": "Technology",
        "JPM": "Financials"
    },
    
    # Position limits
    max_positions=15,
    min_position_size=0.02,
    
    # Turnover control
    max_turnover=0.30,
    current_weights={"AAPL": 0.20, "MSFT": 0.30, ...},
    
    # Per-asset bounds
    per_asset_bounds={
        "AAPL": (0.05, 0.30),  # Min 5%, Max 30%
        "MSFT": (0.10, 0.40)
    }
)

See Constraints Documentation for detailed examples and best practices.

Available Methods

PyOptima provides 21 portfolio optimization methods organized by category:

Efficient Frontier Methods

• min_volatility - Minimize portfolio volatility
• max_sharpe - Maximize Sharpe ratio
• max_quadratic_utility - Maximize quadratic utility
• efficient_risk - Maximize return for target volatility
• efficient_return - Minimize volatility for target return

Black-Litterman Methods

• black_litterman_max_sharpe - BL with max Sharpe objective
• black_litterman_min_volatility - BL with min volatility objective
• black_litterman_quadratic_utility - BL with quadratic utility objective

Conditional Value at Risk (CVaR)

• min_cvar - Minimize CVaR
• efficient_cvar_risk - Maximize return for target CVaR
• efficient_cvar_return - Minimize CVaR for target return

Conditional Drawdown at Risk (CDaR)

• min_cdar - Minimize CDaR
• efficient_cdar_risk - Maximize return for target CDaR
• efficient_cdar_return - Minimize CDaR for target return

Semivariance Methods

• min_semivariance - Minimize downside variance
• efficient_semivariance_risk - Maximize return for target semivariance
• efficient_semivariance_return - Minimize semivariance for target return

Hierarchical Risk Parity (HRP)

• hierarchical_min_volatility - HRP with min volatility
• hierarchical_max_sharpe - HRP with max Sharpe

Critical Line Algorithm (CLA)

• cla_min_volatility - CLA for min volatility
• cla_max_sharpe - CLA for max Sharpe

See examples/ directory for complete, runnable examples of each method.

Data Format Support

PyOptima automatically detects and converts various data formats:

Covariance Matrix Formats

• Nested dict (ETL format): {"matrix": [[...], [...]], "symbols": [...]}
• Flat dict: {"AAPL": {"AAPL": 0.04, "MSFT": 0.01}, ...}
• NumPy array: np.array([[0.04, 0.01], [0.01, 0.05]])
• Pandas DataFrame: pd.DataFrame(..., index=symbols, columns=symbols)

Expected Returns Formats

• Flat dict: {"AAPL": 0.12, "MSFT": 0.11}
• Pandas Series: pd.Series([0.12, 0.11], index=["AAPL", "MSFT"])
• NumPy array: np.array([0.12, 0.11])

Symbols are automatically aligned between covariance matrix and expected returns.

Documentation

• ETL Integration Guide - Using PyOptima with ETL pipeline
• Constraints Documentation - Advanced constraint types and examples
• Usage Guide - Detailed usage examples

API Reference

OptimizationEngine

Main entry point for optimizations.

from pyoptima import OptimizationEngine

engine = OptimizationEngine()

# From file
result = engine.optimize_from_file("config.json")

# From config object
from pyoptima.models.config import OptimizationConfig
config = OptimizationConfig(...)
result = engine.optimize_from_config(config)

# Create reusable optimizer (object factory pattern)
optimizer = engine.create_from_config(config)
result = optimizer.optimize(expected_returns=..., covariance_matrix=...)

Optimizer

Reusable optimizer object created from configuration:

from pyoptima import OptimizationEngine

engine = OptimizationEngine()
optimizer = engine.create_from_file("config.json")

# Reuse with different data
result1 = optimizer.optimize(expected_returns=er1, covariance_matrix=cov1)
result2 = optimizer.optimize(expected_returns=er2, covariance_matrix=cov2)

Result Format

Results are returned as dictionaries:

{
    "weights": {"AAPL": 0.30, "MSFT": 0.40, ...},
    "portfolio_return": 0.12,
    "portfolio_volatility": 0.15,
    "sharpe_ratio": 0.80,
    "status": "optimal",
    "objective_value": 0.04,
    "message": "Optimization successful"
}

ETL Integration

from pyoptima.integration.etl import optimize_from_etl_inputs

# Direct ETL integration
result = optimize_from_etl_inputs(
    consolidated_inputs,
    method="min_volatility",
    constraints=constraints
)

Constraints

from pyoptima.models.config import ConstraintsConfig

constraints = ConstraintsConfig(
    sector_caps={"Technology": 0.40},
    asset_sectors={...},
    max_positions=10,
    max_turnover=0.30,
    current_weights={...},
    per_asset_bounds={...}
)

CLI Usage

Optimization

# Optimize from config file
pyoptima optimize config.json

# With output file
pyoptima optimize config.json --output results.json --pretty

API Server

# Start API server
pyoptima api --host 0.0.0.0 --port 8000

# With auto-reload (development)
pyoptima api --host 0.0.0.0 --port 8000 --reload

UI

# Development mode
pyoptima ui dev

# Build for production
pyoptima ui build

# Serve built UI
pyoptima ui serve

Development Setup

# Clone repository
cd pyoptima

# Install in development mode
pip install -e ".[api,ui]"

# Run tests
pytest tests/

# Run API server
pyoptima api --host 0.0.0.0 --port 8000

# Run UI (development)
pyoptima ui dev

Requirements

• Python 3.10+
• Pyomo (for optimization)
• NumPy, Pandas (for data handling)
• Pydantic >= 2.0.0 (for configuration)
• Optional: FastAPI, Uvicorn (for API server)
• Optional: Next.js, React (for UI)

Project Structure

pyoptima/
├── pyoptima/           # Core package
│   ├── methods/        # Optimization methods (registry)
│   ├── constraints/    # Constraint system
│   ├── adapters/       # Data format adapters
│   ├── integration/   # ETL integration
│   ├── utils/          # Utilities (data formats, Pyomo helpers)
│   └── models/         # Configuration models
├── api/                # FastAPI server (optional)
├── ui/                 # Next.js UI (optional)
├── examples/           # Example configurations and scripts
├── tests/              # Test suite
└── docs/               # Documentation

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

Made with ❤️ for the Python optimization community