Metadata-Version: 2.4
Name: banditbungee
Version: 0.1.1
Summary: A simulation framework for stationary and non-stationary multi-armed bandit experiments.
Author: bandito
License-Expression: MIT
Keywords: bandits,simulation,reinforcement-learning,research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: matplotlib
Requires-Dist: seaborn
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Dynamic: license-file

# BanditBench

A clean, object-oriented simulation framework for stationary and non-stationary multi-armed bandit experiments. 

This framework was designed with a special focus on comparing different forgetting and exploration mechanisms, such as **Sliding-Window UCB** and **Global Discounted UCB**. It includes a variety of dynamic environments to test algorithmic adaptation speeds and robustness against changing reward landscapes.

## Why BanditBench?

BanditBench is not intended to replace large general-purpose bandit libraries. Its goal is to provide a compact, readable, and reproducible framework for studying adaptation in non-stationary stochastic bandit settings.

The package focuses on:
- forgetting mechanisms in UCB-style algorithms
- sudden shifts, smooth drifts, and crossing reward landscapes
- pseudo-regret and adaptation delay metrics
- clean experiment scripts for reproducible comparisons

---

## Features

* **Modular Agents**: Easy-to-extend `Agent` base class. Current implementations include UCB, SW-UCB, and Discounted UCB variants.
* **Dynamic Environments**: Base `Environment` class enforcing strict regret-tracking capabilities. Supports stationary distributions, piecewise sudden shifts, continuous Brownian drifts, and smooth crossing environments.
* **Reproducibility**: Strict adherence to seeded NumPy Random Generators (`default_rng`) for clean, deterministic experimentation across multiple runs.
* **Research-Ready Metrics**: Track instantaneous pseudo-regret, optimal arm selection probabilities, and adaptation delays cleanly across any environment.

---

## Installation

Since this module utilizes standard Python packaging, you can install it easily in your environment.

**Option 1: Install from a local checkout**
```bash
git clone https://github.com/AI-is-fun11/banditbench.git
cd banditbench
pip install -e .
```

**Option 2: Install directly from GitHub after the repository is pushed**
```bash
pip install git+https://github.com/AI-is-fun11/banditbench.git
```

**Option 2b: Install from PyPI after publication**
```bash
pip install banditbungee
```

**Option 3: Install test tooling**
```bash
pip install -e .[dev]
```

---

## Quick Start Example

Below is a minimal example of how to import the module, spin up a stationary environment, and have an agent interact with it.

```python
import numpy as np
from bandits.agents.ucb import UCB1
from bandits.environments.stationary import StationaryBernoulliEnv

# 1. Initialize a 3-arm stationary environment
env = StationaryBernoulliEnv(means=[0.2, 0.5, 0.8], horizon=1000)
env.reset(seed=42)

# 2. Initialize a standard UCB1 agent
agent = UCB1(n_arms=3, c=2.0)

cumulative_regret = 0.0

# 3. Run the interaction loop
for t in range(env.horizon):
    # Agent selects an arm
    chosen_arm = agent.select_arm()
    
    # Environment yields a reward
    reward = env.step(chosen_arm)
    
    # Agent updates its internal statistics
    agent.update(chosen_arm, reward)
    
    # Track pseudo-regret (true best mean - chosen mean)
    instant_regret = env.best_mean() - env.current_means()[chosen_arm]
    cumulative_regret += instant_regret

print(f"Final Cumulative Pseudo-Regret: {cumulative_regret:.2f}")
```

---

## Running Experiments

The repository also includes ready-to-run experiment scripts for comparing algorithms in non-stationary settings.

**Piecewise-stationary benchmark**
```bash
python -m bandits.experiments.piecewise_demo
```

This runs repeated simulations for `DiscountedUCB` and `SlidingWindowUCB`, then saves:
- figures to `figures/piecewise_demo/`
- summary files to `results/piecewise_demo/`

**Crossing cosine benchmark**
```bash
python -m bandits.experiments.cosine_demo
```

This generates a smooth two-arm crossing environment and saves:
- figures to `figures/cosine_demo/`
- summary files to `results/cosine_demo/`

---

## Plotting

Plot helpers live in `bandits/plots/` and operate on the summarized output returned by the experiment utilities.

Example:

```python
from bandits.agents.sw_ucb import SlidingWindowUCB
from bandits.environments.stationary import StationaryBernoulliEnv
from bandits.experiments.multi_run import run_many
from bandits.metrics.summary import summarize_run
from bandits.plots.piecewise_plots import plot_cumulative_regret

raw = run_many(
    agent_factory=lambda: SlidingWindowUCB(n_arms=2, window_size=20),
    env_factory=lambda: StationaryBernoulliEnv(means=[0.4, 0.7], horizon=200),
    seeds=[0, 1, 2, 3, 4],
)

summary = summarize_run(raw)
plot_cumulative_regret({"SlidingWindowUCB": summary}, out_dir="figures/example")
```

The main plotting functions include:
- `plot_cumulative_regret`
- `plot_instantaneous_regret`
- `plot_optimal_tracking`
- `plot_environment`
- `plot_means_path`
- `plot_change_metric_bars`

---

## Directory Structure

* `bandits/agents/`: Bandit algorithm implementations. All agents inherit from `Agent`.
* `bandits/environments/`: Testbeds for both stationary and non-stationary reward distributions. All environments inherit from `Environment`.
* `bandits/experiments/`: Configurable scripts to run large-scale sweeps and comparisons.
* `bandits/metrics/`: Calculation of pseudo-regret, adaptation time, and probability of optimal arm selection.
* `bandits/plots/`: Visualization tools to compare agent performances seamlessly.
* `tests/`: Basic smoke tests.
