Metadata-Version: 2.4
Name: pyoptima
Version: 0.0.6
Summary: Portfolio optimization with a sklearn-style API and config-driven workflows. Multiple objectives, constraints, and solvers.
Author-email: semantic developers <na@example.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/optophi/pyoptima
Project-URL: Documentation, https://optophi.github.io/pyoptima/
Project-URL: Repository, https://github.com/optophi/pyoptima
Project-URL: Issues, https://github.com/optophi/pyoptima/issues
Keywords: optimization,portfolio,mean-variance,sharpe,risk-parity,solver,pyomo
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Mathematics
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pyomo>=6.0.0
Requires-Dist: numpy>=1.20.0
Requires-Dist: pandas>=1.3.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: highspy>=1.0.0
Requires-Dist: idaes-pse>=2.5.0
Provides-Extra: api
Requires-Dist: fastapi>=0.104.0; extra == "api"
Requires-Dist: uvicorn[standard]>=0.24.0; extra == "api"
Requires-Dist: pydantic-settings>=2.0.0; extra == "api"
Requires-Dist: python-multipart>=0.0.6; extra == "api"
Requires-Dist: PyJWT>=2.8.0; extra == "api"
Requires-Dist: httpx>=0.24.0; extra == "api"
Provides-Extra: worker
Requires-Dist: sqlalchemy>=2.0.0; extra == "worker"
Requires-Dist: alembic>=1.13.0; extra == "worker"
Provides-Extra: ui
Requires-Dist: fastapi>=0.104.0; extra == "ui"
Requires-Dist: uvicorn[standard]>=0.24.0; extra == "ui"
Requires-Dist: httpx>=0.24.0; extra == "ui"
Requires-Dist: aiofiles>=23.0.0; extra == "ui"
Provides-Extra: cli
Requires-Dist: rich>=13.0.0; extra == "cli"
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9.0; extra == "postgres"
Provides-Extra: docs
Requires-Dist: mkdocs<2,>=1.5.3; extra == "docs"
Requires-Dist: mkdocs-material>=9.5.0; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.24.0; extra == "docs"
Requires-Dist: pymdown-extensions>=10.0.0; extra == "docs"
Provides-Extra: all
Requires-Dist: pyoptima[api,cli,docs,postgres,ui,worker]; extra == "all"
Provides-Extra: dev
Requires-Dist: pyoptima[all]; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black==26.1.0; extra == "dev"
Requires-Dist: isort==5.13.2; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: build>=0.10.0; extra == "dev"
Requires-Dist: twine>=4.0.0; extra == "dev"
Requires-Dist: jupyterlab>=4.0.0; extra == "dev"
Requires-Dist: ipykernel>=6.0.0; extra == "dev"
Requires-Dist: matplotlib>=3.5.0; extra == "dev"
Dynamic: license-file

# PyOptima

> **Portfolio optimization with a sklearn-style API and config-driven workflows**

[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

**PyOptima** is a Python package for portfolio optimization. Use a composable, sklearn-style API in code or run from YAML/JSON configs. Many objectives (min volatility, max Sharpe, efficient frontier, risk parity, CVaR, etc.), constraints (sector caps, cardinality, turnover), and solvers (IPOPT, HiGHS, others via Pyomo).

## Features

- **sklearn-style API** – `PortfolioOptimizer(objective=..., constraints=[...]).solve(**data)` with `get_params` / `set_params` / `clone`
- **Config-driven** – `PortfolioOptimizer.from_config(path).solve()` or `run_from_config(path)` for any template
- **Objectives** – min_volatility, max_sharpe, efficient_return, efficient_risk, max_utility, risk_parity, min_cvar, Black-Litterman, and more
- **Constraints** – Weight bounds, sector caps/mins, cardinality, turnover, per-asset bounds
- **Solvers** – IPOPT (default for portfolio), HiGHS, CBC, GLPK, Gurobi via Pyomo
- **IO** – `read_portfolio_csv`, `read_portfolio_json`, pandas-friendly result (`weights_to_dataframe()`)
- **ETL** – Batch optimization and ETL output formatting for pipelines

## Installation

### Core Library

```bash
pip install pyoptima
```

### With Optional Dependencies

```bash
# API server
pip install pyoptima[api]

# UI (Next.js frontend)
pip install pyoptima[ui]

# Worker (durable job queue; requires SQLAlchemy)
pip install pyoptima[worker]

# All optional dependencies
pip install pyoptima[api,ui,worker]
```

### Solvers

PyOptima uses Pyomo, which supports multiple solvers. Run `./setup.sh` to install everything (no system packages required):
- **IPOPT** (default for portfolio) - Quadratic/nonlinear optimization (binary via `idaes-pse`, Pyomo's `ipopt_v2` interface)
- **HiGHS** - Linear and mixed-integer programming via `highspy`
- **CBC** - Linear/integer programming (system package)
- **GLPK** - Linear programming (system package)
- **Gurobi** - Commercial solver (requires license)

## Quick Start

### Config-driven (recommended)

Config schema: `template`, `data`, `solver`. For portfolio, `data` includes `objective`, `expected_returns`, `covariance_matrix`.

**1. Create a config file** (`portfolio.json`):

```json
{
  "template": "portfolio",
  "data": {
    "objective": "min_volatility",
    "expected_returns": [0.12, 0.11, 0.15],
    "covariance_matrix": [
      [0.04, 0.01, 0.02],
      [0.01, 0.05, 0.01],
      [0.02, 0.01, 0.06]
    ],
    "symbols": ["AAPL", "MSFT", "GOOGL"],
    "weight_bounds": [0, 1]
  },
  "solver": { "name": "ipopt" }
}
```

**2. Run from Python**

```python
from pyoptima import run_from_config, PortfolioOptimizer, load_config_file

# One-shot (any template: portfolio, knapsack, lp, ...)
result = run_from_config("portfolio.json")  # returns OptimizationResult
print(result.weights, result.portfolio_return, result.portfolio_volatility)

# Or: build optimizer from config, then solve (portfolio only)
opt = PortfolioOptimizer.from_config("portfolio.json")
result = opt.solve()  # uses config data
```

**3. CLI**

```bash
pyoptima solve portfolio.json
```

### Choosing an API

- **Config-only:** `run_from_config(path)` — DAGs, FSM, reproducibility.
- **Code (sklearn-style):** `PortfolioOptimizer(...).solve(**data)` — interactive, pipelines.
- **Generic template:** `solve("knapsack", data)` or `get_template("lp").solve(data)` — LP, knapsack, TSP, etc.
- **Batch/ETL:** `optimize_batch(records, objective=..., **kwargs)` — pycharter, many records.
- **API / CLI:** POST `/v1/optimize` or `pyoptima solve config.json` — remote or scripts.

See [API Quick Reference](docs/API_QUICK_REFERENCE.md). Integration: [pystator](docs/PYSTATOR_INTEGRATION.md), [Airflow](docs/AIRFLOW_INTEGRATION.md).

### Code-only (sklearn-style)

```python
from pyoptima import PortfolioOptimizer

opt = PortfolioOptimizer(objective="min_volatility", max_weight=0.4)
result = opt.solve(
    expected_returns=[0.12, 0.11, 0.15],
    covariance_matrix=[[0.04, 0.01, 0.02], [0.01, 0.05, 0.01], [0.02, 0.01, 0.06]],
    symbols=["AAPL", "MSFT", "GOOGL"],
)
print(result.weights, result.portfolio_return, result.portfolio_volatility)
```

## ETL Integration

PyOptima provides first-class integration with pycharter ETL pipelines. Use `pyoptima.etl.optimize_batch` directly as a pycharter `custom_function`:

### pycharter Configuration (No Bridge Code Needed)

```yaml
# transform.yaml
custom_function:
  module: pyoptima.etl
  function: optimize_batch
  mode: batch
  kwargs:
    objective: min_volatility
    solver: ipopt
```

### Python Usage

```python
from pyoptima.etl import optimize_batch

# ETL input format
data = [{
    "job_id": "growth-2025-01-06",
    "symbols": ["AAPL", "MSFT", "GOOGL"],
    "covariance_matrix": {
        "matrix": [[0.04, 0.01, 0.02], [0.01, 0.05, 0.01], [0.02, 0.01, 0.06]],
        "symbols": ["AAPL", "MSFT", "GOOGL"]
    },
    "expected_returns": {"AAPL": 0.12, "MSFT": 0.11, "GOOGL": 0.15},
}]

# Optimize - returns ETL-ready output
results = optimize_batch(data, objective="min_volatility", solver="ipopt")

# Output format ready for loading
print(results[0]["job_id"])        # "growth-2025-01-06"
print(results[0]["weights"])       # {"AAPL": 0.25, "MSFT": 0.40, "GOOGL": 0.35}
print(results[0]["status"])        # "optimal"
print(results[0]["volatility"])    # 0.142
```

**Key features:** pycharter `custom_function` support, input normalization (dict/list/numpy/pandas), ETL-ready output (job_id, weights, status, metrics), all portfolio objectives, per-record error handling.

See [ETL Integration Guide](docs/ETL_INTEGRATION.md) for complete details.

## Constraints

PyOptima supports advanced portfolio constraints beyond basic weight bounds:

### Available Constraints

- **Sector Caps** - Limit total exposure to specific sectors
- **Sector Minimums** - Enforce minimum exposure to sectors
- **Cardinality** - Maximum number of non-zero positions
- **Turnover Limits** - Control trading from current portfolio state
- **Per-Asset Bounds** - Individual min/max bounds per asset
- **Minimum Position Size** - Conditional holding requirements

### Example

```python
from pyoptima import PortfolioOptimizer, SectorCaps, SumToOne, WeightBounds

opt = PortfolioOptimizer(
    objective="min_volatility",
    constraints=[
        WeightBounds(0.05, 0.40),
        SumToOne(),
        SectorCaps({"Technology": 0.50}, asset_sectors={"AAPL": "Technology", "MSFT": "Technology"}),
    ],
)
result = opt.solve(expected_returns=..., covariance_matrix=..., symbols=...)
```

Or pass constraints as kwargs in config `data`:

```python
# In config data
constraints = dict(
    # Sector constraints
    sector_caps={"Technology": 0.40, "Financials": 0.30},
    asset_sectors={
        "AAPL": "Technology",
        "MSFT": "Technology",
        "JPM": "Financials"
    },
    
    # Position limits
    max_positions=15,
    min_position_size=0.02,
    
    # Turnover control
    max_turnover=0.30,
    current_weights={"AAPL": 0.20, "MSFT": 0.30, ...},
    
    # Per-asset bounds
    per_asset_bounds={
        "AAPL": (0.05, 0.30),  # Min 5%, Max 30%
        "MSFT": (0.10, 0.40)
    }
)
```

See [Constraints Documentation](docs/CONSTRAINTS.md) for detailed examples and best practices.

## Available Methods

PyOptima provides 21 portfolio optimization methods organized by category:

### Efficient Frontier Methods
- `min_volatility` - Minimize portfolio volatility
- `max_sharpe` - Maximize Sharpe ratio
- `max_quadratic_utility` - Maximize quadratic utility
- `efficient_risk` - Maximize return for target volatility
- `efficient_return` - Minimize volatility for target return

### Black-Litterman Methods
- `black_litterman_max_sharpe` - BL with max Sharpe objective
- `black_litterman_min_volatility` - BL with min volatility objective
- `black_litterman_quadratic_utility` - BL with quadratic utility objective

### Conditional Value at Risk (CVaR)
- `min_cvar` - Minimize CVaR
- `efficient_cvar_risk` - Maximize return for target CVaR
- `efficient_cvar_return` - Minimize CVaR for target return

### Conditional Drawdown at Risk (CDaR)
- `min_cdar` - Minimize CDaR
- `efficient_cdar_risk` - Maximize return for target CDaR
- `efficient_cdar_return` - Minimize CDaR for target return

### Semivariance Methods
- `min_semivariance` - Minimize downside variance
- `efficient_semivariance_risk` - Maximize return for target semivariance
- `efficient_semivariance_return` - Minimize semivariance for target return

### Hierarchical Risk Parity (HRP)
- `hierarchical_min_volatility` - HRP with min volatility
- `hierarchical_max_sharpe` - HRP with max Sharpe

### Critical Line Algorithm (CLA)
- `cla_min_volatility` - CLA for min volatility
- `cla_max_sharpe` - CLA for max Sharpe

See `examples/` directory for complete, runnable examples of each method.

## Data Format Support

PyOptima automatically detects and converts various data formats:

### Covariance Matrix Formats
- **Nested dict** (ETL format): `{"matrix": [[...], [...]], "symbols": [...]}`
- **Flat dict**: `{"AAPL": {"AAPL": 0.04, "MSFT": 0.01}, ...}`
- **NumPy array**: `np.array([[0.04, 0.01], [0.01, 0.05]])`
- **Pandas DataFrame**: `pd.DataFrame(..., index=symbols, columns=symbols)`

### Expected Returns Formats
- **Flat dict**: `{"AAPL": 0.12, "MSFT": 0.11}`
- **Pandas Series**: `pd.Series([0.12, 0.11], index=["AAPL", "MSFT"])`
- **NumPy array**: `np.array([0.12, 0.11])`

Symbols are automatically aligned between covariance matrix and expected returns.

## Documentation

- [ETL Integration Guide](docs/ETL_INTEGRATION.md) - Using PyOptima with ETL pipeline
- [Constraints Documentation](docs/CONSTRAINTS.md) - Advanced constraint types and examples
- [Usage Guide](USAGE.md) - Detailed usage examples

## API Reference

### Config-driven API

```python
from pyoptima import run_from_config, load_config_file, OptimizationConfig

# One-shot from path or dict
result = run_from_config("portfolio.json")
config = load_config_file("portfolio.json")

# Config shape: template, data, solver (see Quick Start)
```

### PortfolioOptimizer (sklearn-style + from_config)

```python
from pyoptima import PortfolioOptimizer

# Build from config (portfolio only); solve uses config data
opt = PortfolioOptimizer.from_config("min_volatility.json")
result = opt.solve()

# Or solve with different data
result2 = opt.solve(expected_returns=er2, covariance_matrix=cov2, symbols=symbols)
```

### Result Format

Results are returned as dictionaries:

```python
{
    "weights": {"AAPL": 0.30, "MSFT": 0.40, ...},
    "portfolio_return": 0.12,
    "portfolio_volatility": 0.15,
    "sharpe_ratio": 0.80,
    "status": "optimal",
    "objective_value": 0.04,
    "message": "Optimization successful"
}
```

### ETL Integration

```python
from pyoptima.etl import optimize_batch

# ETL (pycharter custom_function)
results = optimize_batch(
    data=[{"expected_returns": {...}, "covariance_matrix": {...}}],
    objective="min_volatility",
    max_weight=0.40,
    solver="ipopt"
)
```

### Constraints via kwargs

```python
# Pass constraints as kwargs to optimize_batch
results = optimize_batch(
    data,
    objective="min_volatility",
    sector_caps={"Technology": 0.40},
    asset_sectors={"AAPL": "Technology", ...},
    max_assets=10,
    max_turnover=0.30,
    current_weights={"AAPL": 0.20, ...},
    max_weight=0.30,
)
```

## CLI Usage

### Optimization

```bash
# Optimize from config file
pyoptima optimize config.json

# With output file
pyoptima optimize config.json --output results.json --pretty
```

### API Server

```bash
# Start API server
pyoptima api --host 0.0.0.0 --port 8000

# With auto-reload (development)
pyoptima api --host 0.0.0.0 --port 8000 --reload
```

### Worker (durable queue)

When long-running optimizations should survive restarts and scale across multiple processes, use the DB-backed queue and worker:

1. Set `PYOPTIMA_DATABASE_URL` (e.g. `sqlite:///pyoptima.db` or PostgreSQL).
2. Create the table (optional; the worker creates it if missing):
   ```bash
   pyoptima db upgrade
   ```
3. Start one or more workers:
   ```bash
   pyoptima worker --concurrency 5 --poll-interval 500
   ```
4. Start the API with the same `PYOPTIMA_DATABASE_URL`. POST `/api/v1/optimize/async` will enqueue jobs; workers claim and process them. Multiple workers can run; each job is claimed by exactly one worker (atomic claim). Graceful shutdown drains in-flight jobs.

### UI

```bash
# Development mode
pyoptima ui dev

# Build for production
pyoptima ui build

# Serve built UI
pyoptima ui serve
```

## Development Setup

```bash
# Clone repository
cd pyoptima

# Install in development mode
pip install -e ".[api,ui]"

# Run tests
pytest tests/

# Run API server
pyoptima api --host 0.0.0.0 --port 8000

# Run UI (development)
pyoptima ui dev
```

## Requirements

- Python 3.11+
- Pyomo (for optimization)
- NumPy, Pandas (for data handling)
- Pydantic >= 2.0.0 (for configuration)
- Optional: FastAPI, Uvicorn (for API server)
- Optional: Next.js, React (for UI)

## Project Structure

```
pyoptima/
├── pyoptima/           # Core package
│   ├── methods/        # Optimization methods (registry)
│   ├── constraints/    # Constraint system
│   ├── adapters/       # Data format adapters
│   ├── integration/   # ETL integration
│   ├── utils/          # Utilities (data formats, Pyomo helpers)
│   └── models/         # Configuration models
├── api/                # FastAPI server (optional)
├── ui/                 # Next.js UI (optional)
├── examples/           # Example configurations and scripts
├── tests/              # Test suite
└── docs/               # Documentation
```

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

---

**Made with ❤️ for the Python optimization community**

