Metadata-Version: 2.4
Name: POMDPPlanners
Version: 0.6.0
Summary: A Python package for POMDP planning algorithms and environments
Author-email: Yaacov Pariente <yaacovpar@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/yaacovpariente/POMDPPlanners
Project-URL: Bug Reports, https://github.com/yaacovpariente/POMDPPlanners/issues
Project-URL: Documentation, https://yaacovpariente.github.io/POMDPPlanners/
Project-URL: Changelog, https://yaacovpariente.github.io/POMDPPlanners/misc/changelog.html
Keywords: pomdp,planning,reinforcement-learning,monte-carlo-tree-search,artificial-intelligence
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: numpy>=1.21.0
Requires-Dist: ray>=2.0.0
Requires-Dist: pandas>=1.3.0
Requires-Dist: matplotlib>=3.4.0
Requires-Dist: pillow>=10.1.0
Requires-Dist: scipy>=1.7.0
Requires-Dist: gymnasium>=0.29.0
Requires-Dist: anytree>=2.8.0
Requires-Dist: mlflow>=2.0.0
Requires-Dist: redis>=4.0.0
Requires-Dist: seaborn>=0.11.0
Requires-Dist: plotly>=5.0.0
Requires-Dist: diskcache>=5.0.0
Requires-Dist: joblib>=1.0.0
Requires-Dist: tqdm>=4.62.0
Requires-Dist: optuna>=2.0.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: dask>=2023.3.0
Requires-Dist: distributed>=2023.3.0
Requires-Dist: dask_jobqueue>=0.8.0
Requires-Dist: cachey>=0.2.1
Requires-Dist: torch>=2.0.0
Requires-Dist: tensorboard>=2.0.0
Requires-Dist: line_profiler>=4.0.0
Requires-Dist: memory_profiler>=0.60.0
Requires-Dist: psutil>=5.8.0
Requires-Dist: bokeh>=3.0.0
Requires-Dist: numba<0.62,>=0.59
Provides-Extra: dev
Requires-Dist: pylint>=2.17.0; extra == "dev"
Requires-Dist: black==23.12.1; extra == "dev"
Requires-Dist: pyright>=1.1.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: nbval>=0.9.6; extra == "dev"
Requires-Dist: papermill>=2.6.0; extra == "dev"
Requires-Dist: setuptools>=77.0.0; extra == "dev"
Requires-Dist: ipykernel>=6.0.0; extra == "dev"
Requires-Dist: jupyter-client>=7.0.0; extra == "dev"
Requires-Dist: nbformat>=5.0.0; extra == "dev"
Requires-Dist: pytest-benchmark>=4.0.0; extra == "dev"
Requires-Dist: pybind11>=2.12; extra == "dev"
Requires-Dist: build>=1.0.0; extra == "dev"
Requires-Dist: highway-env>=1.12.1; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=5.0.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
Requires-Dist: sphinx-autodoc-typehints>=1.17.0; extra == "docs"
Requires-Dist: myst-parser>=0.18.0; extra == "docs"
Dynamic: license-file

# POMDPPlanners

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/release/python-3100/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

POMDPPlanners is a set of reliable implementations of **POMDP (Partially Observable Markov Decision Process)** planning algorithms and environments in Python. It provides standardized simulation studies for research and production-quality planners for industrial applications — from classic benchmarks like Tiger and RockSample to photorealistic autonomous driving and robotic manipulation.

<p align="center">
  <img src="docs/images/carla_chase_camera.png" alt="CARLA autonomous driving environment rendered by CarlaPOMDP's chase camera" width="49%">
  <img src="docs/images/isaac_lab_franka_reach.png" alt="Isaac Sim Franka reach task rendered by IsaacLabPOMDP's viewport camera" width="49%">
  <img src="docs/images/firefighting.jpg" alt="3D episode replay of the Firefighting environment" width="49%">
  <img src="docs/images/capture_the_flag.jpg" alt="3D episode replay of the Capture the Flag environment" width="49%">
  <img src="docs/images/light_dark.jpg" alt="3D episode replay of the Light-Dark environment" width="49%">
  <img src="docs/images/push.jpg" alt="3D episode replay of the Push environment" width="49%">
</p>
<p align="center">
  <em>Rendered by the package itself. First row: the <a href="POMDPPlanners/environments/carla_pomdp">CARLA</a> driving environment (left) and the <a href="POMDPPlanners/environments/isaac_lab_pomdp">Isaac Sim / IsaacLab</a> Franka reach environment (right). Realistic environments are integrated from the open-source simulators <a href="https://github.com/carla-simulator/carla">CARLA</a> and <a href="https://github.com/isaac-sim/IsaacLab">NVIDIA Isaac Lab</a> — credit to their authors. Below: environments implemented by the package: <a href="POMDPPlanners/environments/firefighting_pomdp">Firefighting</a> and <a href="POMDPPlanners/environments/capture_the_flag_pomdp">Capture the Flag</a> (second row), <a href="POMDPPlanners/environments/light_dark_pomdp">Light-Dark</a> and <a href="POMDPPlanners/environments/push_pomdp">Push</a> (third row).</em>
</p>
<details>
<summary>Show more environments</summary>
<br>
<p align="center">
  <img src="docs/images/tiger.jpg" alt="3D episode replay of the Tiger environment" width="49%">
  <img src="docs/images/rock_sample.jpg" alt="3D episode replay of the RockSample environment" width="49%">
  <img src="docs/images/laser_tag.jpg" alt="3D episode replay of the LaserTag environment" width="49%">
  <img src="docs/images/pacman.jpg" alt="3D episode replay of the PacMan environment" width="49%">
  <img src="docs/images/battleship.jpg" alt="3D episode replay of the Battleship environment" width="49%">
  <img src="docs/images/maze.jpg" alt="3D episode replay of the Maze environment" width="49%">
  <img src="docs/images/t_maze.jpg" alt="3D episode replay of the T-Maze environment" width="49%">
  <img src="docs/images/snake.jpg" alt="3D episode replay of the Snake environment" width="49%">
  <img src="docs/images/chicheck_invaders.jpg" alt="3D episode replay of the Chicheck Invaders environment" width="49%">
  <img src="docs/images/cartpole.jpg" alt="3D episode replay of the CartPole environment" width="49%">
  <img src="docs/images/mountain_car.jpg" alt="3D episode replay of the Mountain Car environment" width="49%">
  <img src="docs/images/safety_ant_velocity.jpg" alt="3D episode replay of the Safety Ant Velocity environment" width="49%">
  <img src="docs/images/racetrack.jpg" alt="3D episode replay of the Racetrack environment" width="49%">
</p>
<p align="center">
  <em>Row by row: <a href="POMDPPlanners/environments/tiger_pomdp">Tiger</a> and <a href="POMDPPlanners/environments/rock_sample_pomdp">RockSample</a>; <a href="POMDPPlanners/environments/laser_tag_pomdp">LaserTag</a> and <a href="POMDPPlanners/environments/pacman_pomdp">PacMan</a>; <a href="POMDPPlanners/environments/battleship_pomdp">Battleship</a> and <a href="POMDPPlanners/environments/maze_pomdp">Maze</a>; <a href="POMDPPlanners/environments/t_maze_pomdp">T-Maze</a> and <a href="POMDPPlanners/environments/snake_pomdp">Snake</a>; <a href="POMDPPlanners/environments/chicheck_invaders_pomdp">Chicheck Invaders</a> and <a href="POMDPPlanners/environments/cartpole_pomdp">CartPole</a>; <a href="POMDPPlanners/environments/mountain_car_pomdp">Mountain Car</a> and <a href="POMDPPlanners/environments/safety_ant_velocity_pomdp">Safety Ant Velocity</a>; <a href="POMDPPlanners/environments/racetrack_pomdp">Racetrack</a>.</em>
</p>
</details>
<details>
<summary>Show planner episode videos</summary>
<br>
<p align="center">
  <img src="docs/images/pft_dpw_firefighting.webp" alt="3D replay of a Firefighting episode planned by PFT-DPW, ending with the fire out" width="49%">
  <img src="docs/images/pft_dpw_capture_the_flag.webp" alt="3D replay of part of a Capture the Flag episode planned by PFT-DPW" width="49%">
</p>
<p align="center">
  <em>Episodes planned online by PFT-DPW and replayed in the package's 3D viewer. Left: <a href="POMDPPlanners/environments/firefighting_pomdp">Firefighting</a>, where the two firefighters put the fire out. Right: <a href="POMDPPlanners/environments/capture_the_flag_pomdp">Capture the Flag</a>, where the blue team defends its half and tags a red attacker; it does not capture the flag in this episode.</em>
</p>
</details>
<details>
<summary>Show environment space types</summary>

Which space types each environment supports. A world with several variants (for example a discrete and a continuous Light-Dark) gets a check for every type one of its variants uses. Isaac Lab's action and observation types depend on the task.

| Environment | Discrete state | Continuous state | Discrete actions | Continuous actions | Discrete obs. | Continuous obs. |
| --- | :-: | :-: | :-: | :-: | :-: | :-: |
| [Tiger](POMDPPlanners/environments/tiger_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [RockSample](POMDPPlanners/environments/rock_sample_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [Battleship](POMDPPlanners/environments/battleship_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [Capture the Flag](POMDPPlanners/environments/capture_the_flag_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ❌ | ✔️ |
| [Chicheck Invaders](POMDPPlanners/environments/chicheck_invaders_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [Firefighting](POMDPPlanners/environments/firefighting_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [Snake](POMDPPlanners/environments/snake_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [PacMan](POMDPPlanners/environments/pacman_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [Sanity](POMDPPlanners/environments/sanity_pomdp) | ✔️ | ❌ | ✔️ | ❌ | ✔️ | ❌ |
| [Maze / T-Maze](POMDPPlanners/environments/maze_pomdp) | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ❌ |
| [Light-Dark](POMDPPlanners/environments/light_dark_pomdp) | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| [LaserTag](POMDPPlanners/environments/laser_tag_pomdp) | ✔️ | ✔️ | ✔️ | ✔️ | ❌ | ✔️ |
| [Push](POMDPPlanners/environments/push_pomdp) | ❌ | ✔️ | ✔️ | ✔️ | ❌ | ✔️ |
| [CartPole](POMDPPlanners/environments/cartpole_pomdp) | ❌ | ✔️ | ✔️ | ❌ | ❌ | ✔️ |
| [Mountain Car](POMDPPlanners/environments/mountain_car_pomdp) | ❌ | ✔️ | ✔️ | ❌ | ❌ | ✔️ |
| [Safety Ant Velocity](POMDPPlanners/environments/safety_ant_velocity_pomdp) | ❌ | ✔️ | ✔️ | ❌ | ❌ | ✔️ |
| [Racetrack](POMDPPlanners/environments/racetrack_pomdp) | ❌ | ✔️ | ✔️ | ❌ | ❌ | ✔️ |
| [CARLA](POMDPPlanners/environments/carla_pomdp) | ❌ | ✔️ | ✔️ | ❌ | ❌ | ✔️ |
| [Isaac Lab](POMDPPlanners/environments/isaac_lab_pomdp) | ❌ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| [nuPlan](POMDPPlanners/environments/nuplan_pomdp) | ❌ | ✔️ | ✔️ | ❌ | ❌ | ✔️ |

</details>

## Main Features

| **Features**                                          | **POMDPPlanners** |
| ----------------------------------------------------- | ----------------- |
| State-of-the-art online POMDP planners                | :heavy_check_mark: |
| Classic benchmarks & realistic simulator environments | :heavy_check_mark: |
| Easy to define custom environments                    | :heavy_check_mark: |
| Rich belief representations                           | :heavy_check_mark: |
| GPU-vectorized planning & belief updates              | :heavy_check_mark: |
| Risk-sensitive (CVaR) & constrained planning          | :heavy_check_mark: |
| Parallel experiment framework with persistent caching | :heavy_check_mark: |
| Hyperparameter tuning (Optuna)                        | :heavy_check_mark: |
| Progress tracking & Slack notifications               | :heavy_check_mark: |
| Tutorial notebooks                                    | :heavy_check_mark: |
| Documentation                                         | :heavy_check_mark: |
| Comprehensive test suite & type hints                 | :heavy_check_mark: |

## Documentation

Documentation is available online: https://yaacovpariente.github.io/POMDPPlanners/

## Installation

**Note:** POMDPPlanners requires Python 3.10+.

Install the latest release from PyPI:

```bash
pip install POMDPPlanners
```

The release is a source distribution, so pip compiles the package's C++ modules and needs a C++ compiler.

Or install from source:

```bash
# Clone the repository
git clone https://github.com/yaacovpariente/POMDPPlanners.git
cd POMDPPlanners

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install the package
pip install -e .
```

## Example

Plan with POMCP on the classic Tiger problem in just a few lines:

```python
from POMDPPlanners.environments.tiger_pomdp import TigerPOMDP
from POMDPPlanners.planners.mcts_planners.pomcp import POMCP
from POMDPPlanners.utils.belief_factory import create_environment_belief

env = TigerPOMDP(discount_factor=0.95)
planner = POMCP(environment=env, discount_factor=0.95, depth=20,
                exploration_constant=10.0, n_simulations=1000,
                name="POMCP")
belief = create_environment_belief(env, n_particles=200)

actions, _ = planner.action(belief)
print(f"Recommended action: {actions[0]}")
```

## Running Planner Evaluation Experiment

The recommended entry point for end-to-end experiments is `LocalSimulationsAPI`,
which runs parallel episodes, applies persistent caching, and returns aggregated
statistics (mean return, CVaR, VaR, confidence intervals).

The example below compares POMCPOW and PFT-DPW on Light-Dark: a robot must
reach a goal, but its position readings are only precise near a light source,
so it has to detour toward the light before it can find the goal. Each planner
gets 2 seconds per decision, and the episodes (30 per planner) run in parallel
on every CPU core.
Save it as `compare_planners.py` and run `python compare_planners.py`.

```python
from pathlib import Path

from POMDPPlanners.environments import ContinuousLightDarkPOMDPDiscreteActions
from POMDPPlanners.planners.mcts_planners.pomcpow import POMCPOW
from POMDPPlanners.planners.mcts_planners.pft_dpw import PFT_DPW
from POMDPPlanners.utils.action_samplers import DiscreteActionSampler
from POMDPPlanners.utils.belief_factory import create_environment_belief
from POMDPPlanners.simulations.simulation_apis.local_simulations_api import LocalSimulationsAPI
from POMDPPlanners.core.simulation import EnvironmentRunParams

env = ContinuousLightDarkPOMDPDiscreteActions(discount_factor=0.95)
sampler = DiscreteActionSampler(env.get_actions())

pomcpow = POMCPOW(environment=env, discount_factor=0.95, depth=10,
                  exploration_constant=200.0, k_o=1.0, k_a=1.0,
                  alpha_o=0.5, alpha_a=0.5, time_out_in_seconds=2,
                  action_sampler=sampler, name="POMCPOW")
pft_dpw = PFT_DPW(environment=env, discount_factor=0.95, depth=10,
                  exploration_constant=200.0, time_out_in_seconds=2,
                  action_sampler=sampler, name="PFT_DPW")
belief = create_environment_belief(env, n_particles=200)

api = LocalSimulationsAPI()
_, stats = api.run_multiple_environments_and_policies(
    environment_run_params=[
        EnvironmentRunParams(
            environment=env,
            belief=belief,
            policies=[pomcpow, pft_dpw],
            num_episodes=30,
            num_steps=30,
        )
    ],
    alpha=0.1,
    confidence_interval_level=0.95,
    experiment_name="LightDark_Evaluation",
    n_jobs=-1,  # run episodes in parallel, one per CPU core
    cache_dir_path=Path("results"),
)
print(stats[["policy", "average_return", "task_completion_rate"]])
```

### Viewing the results

The run writes its results under `results/`. Open them in the local results
site:

```bash
pomdp-report serve results --port 8765
```

Then browse to http://127.0.0.1:8765, using the port you passed to `--port`.
Open the `LightDark_Evaluation` experiment to compare the two planners side by
side on expected return and task completion rate, and replay any episode in 3D.

The video below walks through the whole example: copying the code from this
README, running it, and comparing the planners in the results site. It also
shows the List and Table views, editing a chart's title and axes, exporting
the chart as an image, and playing every episode at once with Live.

https://github.com/user-attachments/assets/3b305863-5180-454d-bb47-0a9dd13d76cf

Long-running experiments can report progress to Slack and a local progress
database, including detection of crashed or stalled runs — set
`SLACK_WEBHOOK_URL` in your environment and notifications are picked up
automatically. See
[`NotificationConfig`](POMDPPlanners/simulations/simulations_deployment/run_progress/config.py)
for details.

### Parameter tuning

`LocalSimulationsAPI.run_optimize_and_evaluate` tunes each planner with an
Optuna study and evaluates the tuned planners over multiple episodes.

The example below tunes POMCPOW and PFT-DPW on two environments, RockSample
and Push, and compares the tuned planners on each. Everything the study uses is
written out so you can edit it in place: the search range of each parameter,
the planner settings that stay fixed, and the objectives each trial is scored
on (average return and task completion rate). Early stopping ends a study once
its best trials stop improving. Each planner's name carries its environment,
because the evaluation requires planner names to be unique across environments.
Save it as `tune_planners.py` and run `python tune_planners.py`.

```python
from pathlib import Path

from POMDPPlanners.environments import PushPOMDP, RockSamplePOMDP
from POMDPPlanners.planners.mcts_planners.pft_dpw import PFT_DPW
from POMDPPlanners.planners.mcts_planners.pomcpow import POMCPOW
from POMDPPlanners.core.simulation.hyperparameter_tuning import (
    EarlyStoppingConfig,
    HyperParameterOptimizationDirection,
    HyperParameterRunParams,
    HyperParamPlannerConfig,
    NumericalHyperParameter,
)
from POMDPPlanners.simulations.simulation_apis.local_simulations_api import LocalSimulationsAPI
from POMDPPlanners.utils.action_samplers import DiscreteActionSampler
from POMDPPlanners.utils.belief_factory import create_environment_belief

N_TRIALS = 50            # at most this many Optuna trials per planner
EPISODES_PER_TRIAL = 20  # episodes that score one trial
EVAL_EPISODES = 30       # fresh episodes for each tuned planner
NUM_STEPS = 30           # step limit per episode
DISCOUNT_FACTOR = 0.95
DEPTH = 10               # search depth, fixed rather than tuned

# What each trial is scored on, and in which direction.
OBJECTIVES = [
    ("average_return", HyperParameterOptimizationDirection.MAXIMIZE),
    ("task_completion_rate", HyperParameterOptimizationDirection.MAXIMIZE),
]


def search_space(env):
    max_exploration = (env.reward_range[1] - env.reward_range[0]) * DEPTH
    return [
        NumericalHyperParameter(0.0, max_exploration, "exploration_constant"),  # UCB exploration
        NumericalHyperParameter(1, 10, "k_a"),  # action widening coefficient
        NumericalHyperParameter(0.01, 0.5, "alpha_a"),  # action widening exponent
        NumericalHyperParameter(1, 10, "k_o"),  # observation widening coefficient
        NumericalHyperParameter(0.01, 0.5, "alpha_o"),  # observation widening exponent
    ]


studies = []
for env in (
    RockSamplePOMDP(discount_factor=DISCOUNT_FACTOR),
    PushPOMDP(discount_factor=DISCOUNT_FACTOR),
):
    belief = create_environment_belief(env, n_particles=200)
    sampler = DiscreteActionSampler(env.get_actions())
    for policy_cls, name in ((POMCPOW, "POMCPOW"), (PFT_DPW, "PFT_DPW")):
        planner = HyperParamPlannerConfig(
            policy_cls=policy_cls,
            hyper_parameters=search_space(env),
            constant_parameters={  # settings that stay fixed in every trial
                "discount_factor": DISCOUNT_FACTOR,
                "depth": DEPTH,
                "name": f"{name}_{env.name}",
                "environment": env,
                "action_sampler": sampler,
                "time_out_in_seconds": 1,
            },
        )
        studies.append(
            HyperParameterRunParams(
                environment=env,
                belief=belief,
                hyper_param_planner_config=planner,
                num_episodes=EPISODES_PER_TRIAL,
                num_steps=NUM_STEPS,
                n_trials=N_TRIALS,
                parameters_to_optimize=OBJECTIVES,
                early_stopping=EarlyStoppingConfig(patience=20, min_trials=20),
            )
        )

api = LocalSimulationsAPI()
_, stats = api.run_optimize_and_evaluate(
    configs=studies,
    evaluation_episodes=EVAL_EPISODES,
    evaluation_steps=NUM_STEPS,
    optimization_n_jobs=-1,  # use every CPU core
    evaluation_n_jobs=-1,
    experiment_name="Tuning_RockSample_Push",
    cache_dir_path=Path("results"),
)
print(stats[["environment", "policy", "average_return", "task_completion_rate"]])
```

These constants allow up to 1,000 tuning episodes per planner (50 × 20), then
30 evaluation episodes for each tuned planner. Early stopping ends a planner's study once 20 trials in a row
fail to improve its best results, but never before 20 trials have run, so
`N_TRIALS` is an upper bound.

In `pomdp-report serve results`, the `Tuning_RockSample_Push` experiment shows
the whole study as one card. Its page has a "Compare Tuned planners" block for
each environment, listing every evaluation metric with its confidence interval
and filters for metrics, planners and environments. Each tuned planner opens a
tuning view with its trials, diagnostic charts and evaluation episodes, and
both pages can build a figure and download it as SVG, PNG or CSV.

The video below walks through the whole example: copying the code from this
README, running the full study, and reading the results on PushPOMDP in the
results site. It shows the planner comparison and its filters, a tuning view
with its trials and diagnostic charts, and the evaluation episodes played Live.

https://github.com/user-attachments/assets/1d0f551d-d2f1-48f1-83b7-af31c5fd2778

## Tutorial Notebooks

Self-contained Jupyter notebooks with executable end-to-end examples live in
[`docs/examples/`](docs/examples/):

| Notebook | What it covers |
|---|---|
| [`basic_usage.ipynb`](docs/examples/basic_usage.ipynb) | Environment setup, belief initialization, single-planner evaluation |
| [`planners_comparison.ipynb`](docs/examples/planners_comparison.ipynb) | Side-by-side comparison of POMCP / POMCPOW / PFT-DPW on a shared environment |
| [`belief_representations.ipynb`](docs/examples/belief_representations.ipynb) | Particle, Gaussian, and Gaussian-mixture beliefs |
| [`hyperparameter_tuning.ipynb`](docs/examples/hyperparameter_tuning.ipynb) | End-to-end Optuna search via `run_optimize_and_evaluate` |
| [`advanced_optimization.ipynb`](docs/examples/advanced_optimization.ipynb) | Multi-config tuning, custom search spaces |
| [`custom_environment.ipynb`](docs/examples/custom_environment.ipynb) | Implementing a new `Environment` subclass |
| [`tree_analysis_debugging.ipynb`](docs/examples/tree_analysis_debugging.ipynb) | Inspecting and debugging search trees |

## Implemented Algorithms

| **Algorithm** | **Description** |
| ------------- | --------------- |
| POMCP | Monte Carlo tree search with unweighted particle beliefs (Silver & Veness, 2010) |
| POMCP-DPW | POMCP with double progressive widening for large action/observation spaces |
| POMCPOW | Weighted-particle MCTS for continuous observation spaces (Sunberg & Kochenderfer, 2018) |
| PFT-DPW | Particle filter tree with double progressive widening (Sunberg & Kochenderfer, 2018) |
| Sparse PFT | Particle filter tree with sparse observation branching |
| Sparse Sampling | Depth-limited sparse sampling of the belief MDP (Kearns et al., 2002) |
| BetaZero | Neural-network-guided belief-state MCTS with learned policy and value |
| ConstrainedZero | Safety-constrained variant of BetaZero |
| Constrained POMCPOW / Constrained PFT-DPW | Cost-constrained online planning |
| iCVaR POMCPOW / iCVaR PFT-DPW / iCVaR Sparse Sampling | Risk-averse planning with iterated CVaR objectives |
| VOPP | Fully GPU-vectorized online POMDP planning (Hoerger et al., 2025) |
| Discrete Action Sequences | Open-loop baseline planner |

## Implemented Environments

| **Environment** | **Description** |
| --------------- | --------------- |
| Tiger | Classic information-gathering benchmark |
| Light-Dark | Navigation under state-dependent observation noise (continuous & discrete variants) |
| RockSample | Rover science mission with sensing trade-offs |
| LaserTag | Pursuit with laser range-finder observations |
| PacMan | Arcade-style pursuit-evasion with rendering |
| Battleship | Probe a hidden fleet one cell at a time and sink every ship in as few probes as possible |
| Maze / T-Maze | Memory task: a one-time noisy cue at the start says which distant goal pays (T-shaped corridor, or a generated maze with discrete & continuous movement) |
| Snake | Arcade Snake with hidden food, located through a short-range vision window and a noisy scent |
| Chicheck Invaders | Grid shooter: clear a flock of chickens whose dives start unseen |
| CartPole / MountainCar | Partially observable versions of the Gym classics |
| Push | Object manipulation under contact uncertainty |
| Firefighting | Several firefighters put out a grid fire spread by a hidden wind |
| Capture the Flag | Team game against hidden opponents and a hidden flag, sensed through noisy range readings |
| Safety-Ant-Velocity | Safety-constrained quadruped locomotion |
| Racetrack | Racing on [HighwayEnv](https://github.com/Farama-Foundation/HighwayEnv)'s racetrack, with matched fully and partially observed modes |
| CARLA | Photorealistic autonomous driving in the [CARLA](https://github.com/carla-simulator/carla) simulator |
| Isaac Lab | Franka reach manipulation in [NVIDIA Isaac Lab](https://github.com/isaac-sim/IsaacLab) / Isaac Sim |
| nuPlan | Autonomous driving planning on real-world driving logs |
| Sanity | Minimal environment for quick sanity checks |

Custom environments are first-class: subclass `Environment`, implement the
transition, observation, and reward models, and every planner and the whole
experiment framework work with it out of the box. See
[`custom_environment.ipynb`](docs/examples/custom_environment.ipynb).

## Belief Representations

Beliefs are pluggable and planner-independent: unweighted and weighted particle
filters, batched particle beliefs, Gaussian and Gaussian-mixture beliefs, and
GPU-vectorized particle belief updaters for large-scale simulation.

## Citing the Project

If you use POMDPPlanners in your research, please cite:

```bibtex
@misc{pariente2026pomdpplannersopensourcepackagepomdp,
      title={POMDPPlanners: Open-Source Package for POMDP Planning}, 
      author={Yaacov Pariente and Vadim Indelman},
      year={2026},
      eprint={2602.20810},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2602.20810}, 
}
```

## Contributing & Support

Questions, bug reports, and feature requests are welcome on the
[issue tracker](https://github.com/yaacovpariente/POMDPPlanners/issues).

## License

This project is licensed under the MIT License — see the [LICENSE.md](LICENSE.md) file for details.
