Metadata-Version: 2.4
Name: POMDPPlanners
Version: 0.4.0
Summary: A Python package for POMDP planning algorithms and environments
Author-email: Yaacov Pariente <yaacovpar@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/yaacovpariente/POMDPPlanners
Project-URL: Bug Reports, https://github.com/yaacovpariente/POMDPPlanners/issues
Project-URL: Documentation, https://yaacovpariente.github.io/POMDPPlanners/
Keywords: pomdp,planning,reinforcement-learning,monte-carlo-tree-search,artificial-intelligence
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: numpy>=1.21.0
Requires-Dist: ray>=2.0.0
Requires-Dist: pandas>=1.3.0
Requires-Dist: matplotlib>=3.4.0
Requires-Dist: scipy>=1.7.0
Requires-Dist: gymnasium>=0.29.0
Requires-Dist: anytree>=2.8.0
Requires-Dist: mlflow>=2.0.0
Requires-Dist: redis>=4.0.0
Requires-Dist: seaborn>=0.11.0
Requires-Dist: plotly>=5.0.0
Requires-Dist: diskcache>=5.0.0
Requires-Dist: joblib>=1.0.0
Requires-Dist: tqdm>=4.62.0
Requires-Dist: optuna>=2.0.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: dask>=2023.3.0
Requires-Dist: distributed>=2023.3.0
Requires-Dist: dask_jobqueue>=0.8.0
Requires-Dist: cachey>=0.2.1
Requires-Dist: torch>=2.0.0
Requires-Dist: tensorboard>=2.0.0
Requires-Dist: line_profiler>=4.0.0
Requires-Dist: memory_profiler>=0.60.0
Requires-Dist: psutil>=5.8.0
Requires-Dist: bokeh>=3.0.0
Requires-Dist: numba<0.62,>=0.59
Provides-Extra: dev
Requires-Dist: pylint>=2.17.0; extra == "dev"
Requires-Dist: black==23.12.1; extra == "dev"
Requires-Dist: pyright>=1.1.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: nbval>=0.9.6; extra == "dev"
Requires-Dist: papermill>=2.6.0; extra == "dev"
Requires-Dist: setuptools>=77.0.0; extra == "dev"
Requires-Dist: ipykernel>=6.0.0; extra == "dev"
Requires-Dist: jupyter-client>=7.0.0; extra == "dev"
Requires-Dist: nbformat>=5.0.0; extra == "dev"
Requires-Dist: pytest-benchmark>=4.0.0; extra == "dev"
Requires-Dist: pybind11>=2.12; extra == "dev"
Requires-Dist: build>=1.0.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=5.0.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
Requires-Dist: sphinx-autodoc-typehints>=1.17.0; extra == "docs"
Requires-Dist: myst-parser>=0.18.0; extra == "docs"
Dynamic: license-file

# POMDPPlanners

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/release/python-3100/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

A comprehensive Python package for **POMDP (Partially Observable Markov Decision Process)** planning algorithms and environments. POMDPPlanners provides standardized simulation studies for research and reliable implementations of planning algorithms for industrial applications.

## 🎯 Key Features

- **Comprehensive Algorithm Library**: Implementations of state-of-the-art POMDP planning algorithms including POMCP, POMCPOW, POMCP-DPW, PFT-DPW, Sparse PFT, BetaZero, ConstrainedZero, and more
- **Rich Environment Collection**: Classic and modern POMDP environments (Tiger, Light-Dark, RockSample, LaserTag, PacMan, CartPole, Push, Safety-Ant-Velocity, etc.)
- **Flexible Belief Representations**: Particle filters, weighted beliefs, Gaussian beliefs, Gaussian mixture beliefs, and vectorized belief updaters
- **Simulation Framework**: Complete experiment management with hyperparameter tuning, high-level evaluation workflows, and distributed computing support
- **Visualization Tools**: Built-in plotting and visualization capabilities for analysis and debugging
- **Production Ready**: Designed for both research experiments and industrial applications

## 🚀 Quick Start

### Installation

```bash
# Clone the repository
git clone https://github.com/yaacovpariente/POMDPPlanners.git
cd POMDPPlanners

# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install package (standard)
pip install -e .

# Install with development dependencies
pip install -e ".[dev]"
```

### Basic Usage

```python
from POMDPPlanners.environments.tiger_pomdp import TigerPOMDP
from POMDPPlanners.planners.mcts_planners.pomcp import POMCP
from POMDPPlanners.utils.belief_factory import create_environment_belief

env = TigerPOMDP(discount_factor=0.95)
planner = POMCP(environment=env, discount_factor=0.95, depth=20,
                exploration_constant=10.0, n_simulations=1000,
                name="POMCP")
belief = create_environment_belief(env, n_particles=200)

actions, _ = planner.action(belief)
print(f"Recommended action: {actions[0]}")
```

See [Running Experiments](#-running-experiments) below for a parallel
multi-policy evaluation example.

## 📊 Running Experiments

The recommended entry point for end-to-end experiments is `LocalSimulationsAPI`,
which runs parallel episodes, applies persistent caching, and returns aggregated
statistics (mean return, CVaR, VaR, confidence intervals).

```python
from POMDPPlanners.environments import ContinuousLightDarkPOMDPDiscreteActions
from POMDPPlanners.planners.mcts_planners.pomcpow import POMCPOW
from POMDPPlanners.planners.mcts_planners.pft_dpw import PFT_DPW
from POMDPPlanners.utils.action_samplers import DiscreteActionSampler
from POMDPPlanners.utils.belief_factory import create_environment_belief
from POMDPPlanners.simulations.simulation_apis.local_simulations_api import LocalSimulationsAPI
from POMDPPlanners.core.simulation import EnvironmentRunParams

env = ContinuousLightDarkPOMDPDiscreteActions(discount_factor=0.95)
sampler = DiscreteActionSampler(env.get_actions())

pomcpow = POMCPOW(environment=env, discount_factor=0.95, depth=10,
                  exploration_constant=10.0, k_o=2.0, k_a=2.0,
                  alpha_o=0.5, alpha_a=0.5, n_simulations=500,
                  action_sampler=sampler, name="POMCPOW")
pft_dpw = PFT_DPW(environment=env, discount_factor=0.95, depth=10,
                  exploration_constant=10.0, n_simulations=500,
                  action_sampler=sampler, name="PFT_DPW")
belief = create_environment_belief(env, n_particles=200)

api = LocalSimulationsAPI()
_, stats = api.run_multiple_environments_and_policies(
    environment_run_params=[EnvironmentRunParams(
        environment=env, belief=belief,
        policies=[pomcpow, pft_dpw], num_episodes=100, num_steps=30)],
    alpha=0.1, confidence_interval_level=0.95,
    experiment_name="LightDark_Evaluation",
)
```

For hyperparameter search, `LocalSimulationsAPI.run_optimize_and_evaluate(...)`
accepts `HyperParameterRunParams` with Optuna search ranges and forwards the
best configuration to evaluation automatically.

### Progress Tracking and Slack Notifications

Long-running experiments emit lifecycle events (`run_started`,
`episode_completed` heartbeat, `run_finished`, `run_failed`) to a local
SQLite progress DB and, optionally, to Slack. Export `SLACK_WEBHOOK_URL`
before constructing the API and notifications are picked up automatically;
for per-instance control (e.g. routing two parallel simulations to
different channels), pass a `NotificationConfig` directly. An external
watcher CLI catches hard process death (SIGKILL / OOM / reboot) by
monitoring heartbeat age. See
[`NotificationConfig`](POMDPPlanners/simulations/simulations_deployment/run_progress/config.py)
for the full env-var list and watcher invocation.

### Tutorial Notebooks

Self-contained Jupyter notebooks with executable end-to-end examples live in
[`docs/examples/`](docs/examples/):

| Notebook | What it covers |
|---|---|
| [`basic_usage.ipynb`](docs/examples/basic_usage.ipynb) | Environment setup, belief initialization, single-planner evaluation |
| [`planners_comparison.ipynb`](docs/examples/planners_comparison.ipynb) | Side-by-side comparison of POMCP / POMCPOW / PFT-DPW on a shared environment |
| [`belief_representations.ipynb`](docs/examples/belief_representations.ipynb) | Particle, Gaussian, and Gaussian-mixture beliefs |
| [`hyperparameter_tuning.ipynb`](docs/examples/hyperparameter_tuning.ipynb) | End-to-end Optuna search via `run_optimize_and_evaluate` |
| [`advanced_optimization.ipynb`](docs/examples/advanced_optimization.ipynb) | Multi-config tuning, custom search spaces |
| [`custom_environment.ipynb`](docs/examples/custom_environment.ipynb) | Implementing a new `Environment` subclass |

## 🧪 Testing

Run the comprehensive test suite:

```bash
# Activate virtual environment
source .venv/bin/activate

# Run all tests
pytest

# Run specific test categories
pytest POMDPPlanners/tests/test_core/
pytest POMDPPlanners/tests/test_environments/
pytest POMDPPlanners/tests/test_planners/

# Run with verbose output
pytest -v

# Run specific test file
pytest POMDPPlanners/tests/test_core/test_belief.py
```

## 🔧 Development

### Code Quality

```bash
# Format code
black .

# Type checking
python -m pyright POMDPPlanners/

# Run linting
pylint POMDPPlanners/
flake8 .

# Install pre-commit hooks
pre-commit install
```

### Virtual Environment

**Important**: Always activate the virtual environment before development:

```bash
source .venv/bin/activate  # Linux/Mac
# .venv\Scripts\activate   # Windows
```

All commands should be run within this environment for consistent dependency management.

## 📚 Documentation

Comprehensive documentation is generated from docstrings using Sphinx:

```bash
# Build documentation
cd docs/
sphinx-build -b html . _build/html

# Serve locally
python -m http.server 8000 -d _build/html
```

Visit the documentation at: [Project Documentation](https://yaacovpariente.github.io/POMDPPlanners/)

## 📄 License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details.

## 🎓 Citation

If you use POMDPPlanners in your research, please cite:

```bibtex
@misc{pariente2026pomdpplannersopensourcepackagepomdp,
      title={POMDPPlanners: Open-Source Package for POMDP Planning}, 
      author={Yaacov Pariente and Vadim Indelman},
      year={2026},
      eprint={2602.20810},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2602.20810}, 
}
```

## 🛠️ Requirements

- Python 3.10 or higher
- Core dependencies managed via `pyproject.toml` (`pip install -e .`)
- Development dependencies: `pip install -e ".[dev]"`

---
