Metadata-Version: 2.4
Name: aniate
Version: 2.0.0
Summary: Approximate Numerics in Applied Transition Environments: define, simulate, check, solve and plot MDPs and non-Markovian decision problems.
Author: Kabir Murjani
License-Expression: MIT
Project-URL: Source, https://github.com/Kcbir/aniate
Project-URL: Guide, https://github.com/Kcbir/aniate/blob/main/tree.md
Keywords: mdp,reinforcement-learning,markov-decision-process,simulation,non-markovian,reward-machine,gym,dynamic-programming,monte-carlo
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Education
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Mathematics
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.12
Provides-Extra: vis
Requires-Dist: matplotlib>=3.8; extra == "vis"
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Requires-Dist: hypothesis>=6; extra == "test"
Provides-Extra: dev
Requires-Dist: aniate[test,vis]; extra == "dev"
Dynamic: license-file

# aniate

**Approximate Numerics in Applied Transition Environments**

aniate is a Python library for finite decision problems. It covers Markov decision processes (MDPs) and problems in which the reward depends on the history of the episode.

The user specifies the states, actions, transition probabilities, rewards and discount factor. The library then:

- validates the model when it is built
- solves it exactly
- simulates it
- tests other simulators against it
- writes one log line per operation
- saves plots as PDF files

A guide to choosing the right function for a task is in [tree.md](https://github.com/Kcbir/aniate/blob/main/tree.md).

## Installation

```bash
pip install aniate            # numpy and scipy
pip install 'aniate[vis]'     # adds matplotlib, required for plots
```

Python 3.10 or later is required.

## Example

```python
import aniate as an
from aniate import vis

m = an.from_functions(
    states=range(5), actions=["left", "right"],
    transition=lambda s, a: {max(s - 1, 0): 1.0} if a == "left" else {min(s + 1, 4): 0.8, s: 0.2},
    reward=lambda s, a, s_next: 10.0 if s_next == 4 else -1.0,
    gamma=0.9, terminal=[4], initial=0,
)

sol = m.solve()                                   # optimal policy, its exact value, an error bound
runs = an.simulate(m, sol, episodes=10_000)       # 10,000 episodes, run in parallel
report = an.check(m, policy=sol)                  # compares simulated behaviour with the model
vis.save(vis.overview(runs), "overview.pdf")      # four standard plots in one PDF
m.describe()                                      # the model as a JSON-compatible dict
```

A complete workflow uses one constructor and four functions: `solve`, `simulate`, `check` and `vis.save`.

Each operation writes one line to standard error:

```
aniate | model    | 5 states | 2 actions | gamma 0.9 | horizon inf | 14 transitions | valid
aniate | solve    | policy_iteration | 4 iterations | 2.8 ms | start value 3.20876 | bound 8.9e-15
aniate | simulate | 10,000 episodes | 49,768 steps | 4 ms | mean return 3.23989 +/- 0.014 | 10,000 terminated | 0 truncated
aniate | check    | PASSED | 300 episodes | 1,505 steps | 4 pairs tested | return 3.1863 vs model 3.2088
aniate | vis      | saved overview.pdf
```

## Model validation

A model is validated at construction. Construction fails, and the error names each offending state and action, if any of the following hold:

- a row of transition probabilities does not sum to 1
- a state-action pair has no successor
- a terminal state moves to another state or pays a reward
- the discount factor is 1 and no terminal state is reachable

Unreachable states and rewards on transitions of probability zero produce warnings.

`an.check` covers the case where a separate simulator exists. It runs episodes through that simulator and compares the observed transitions, rewards, terminations and returns with the model, using chi-square tests with a Bonferroni correction.

These guarantees rest on runtime checks and a test suite. They have not been formally proved.

## History-dependent rewards

A task such as "visit A before B" is expressed with event labels and an automaton. aniate builds the exact product model and solves it.

```python
from aniate import nmdp

world  = an.problems.gridworld(rows=5, cols=5, goals=((0, 4),), start=(4, 0))
labels = nmdp.Labels(world).at("r4c4", "A").at("r0c4", "B")
task   = an.NMDP(world, labels, nmdp.Ordering(["A", "B"], violation_penalty=-1))
policy = task.solve()        # a MemoryPolicy: policy.step(state) returns the next action
```

## Main functions

| call | purpose |
|---|---|
| `an.from_functions(states, actions, transition, reward, gamma, terminal=, initial=)` | define a model with Python functions |
| `an.Builder(gamma)` | define a model one transition at a time |
| `an.MDP(P, R, gamma)` | define a model from matrices |
| `m.solve()` | optimal policy and exact value |
| `an.evaluate(m, policy)` | exact value of any policy |
| `an.simulate(m, policy, episodes)` | batched episodes, returned as `Episodes` |
| `m.env()` | a Gym-style environment with `reset` and `step` |
| `an.check(m, env=simulator, policy=)` | statistical comparison of a simulator with the model |
| `an.NMDP(world, labels, automaton)` | a problem whose reward depends on history |
| `vis.overview(runs)`, `vis.graph(m)`, `vis.grid(m, sol)` | plots; `vis.save(fig, "name.pdf")` writes a PDF |
| `obj.describe()` | a JSON-compatible summary of any model, solution, report or set of episodes |
| `an.verbosity("quiet")` | set the log level |

## Worked examples

Each folder in `tests/` contains a model, two tests and a script that writes PDF plots.

| folder | problem | result verified by the tests |
|---|---|---|
| `tests/stsp` | stochastic travelling salesman: four customers, roads blocked at random | the exact solution equals the best of all 24 tours (expected cost 17.6227) |
| `tests/gaussian` | a Gaussian random walk between a pool and a pizza | P(pizza first) is 0.5 by symmetry; Monte Carlo agrees; wind raises it to 0.956 |
| `tests/heavy_tail` | a warehouse robot with power-law jam durations | the value matches the closed form; the value martingale has constant mean; an incorrect simulator is detected |

```bash
pytest                               # all tests
python tests/stsp/plot.py            # writes route.pdf and overview.pdf into tests/stsp
```

## Import banner

Importing aniate prints a Fibonacci rectangle to standard error once per process. Set `ANIATE_BANNER=0` to disable it. `python -m aniate` prints it together with the version.

## Development

```bash
pip install -e '.[dev]'
pytest
```

Released under the MIT License.
