Metadata-Version: 2.4
Name: neuralpy-learn
Version: 0.1.0
Summary: Beginner-friendly neural network reinforcement learning in pure NumPy
Author: Neuralpy contributors
License-Expression: MIT
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Dynamic: license-file

# Neuralpy — teach a neural network by letting it try

`neuralpy` is a small Python package for training a **real, fully connected neural network from scratch** with reinforcement learning. The learner is a Deep Q-Network (DQN) with experience replay and a target network. It uses NumPy for matrix arithmetic; **no pretrained model, API key, or internet connection is required to train**.

This is version 0.1.0, designed to be understandable and hackable.

## Install

With Python 3.10 or newer, open a terminal in this folder:

```bash
python -m pip install -e .
```

Run the included working game:

```bash
python examples/treasure_game.py
```

This trains from scratch, reports progress, tests 100 new games, and saves `treasure_brain.npz`.

## First steps

```python
from neuralpy import Brain

# Actions that YOUR game recognizes; no open-source AI model involved.
brain = Brain(actions=["left", "right"], goal=my_reward_function)

# game.reset() starts a level and returns a numeric list, e.g. [0.25, 0.80]
# game.step(action) updates the game and returns (new_list, finished, info_dict)
history = brain.learn(game, episodes=500, max_steps=100)

brain.save("my_brain")
result = brain.play(game)
print(result)
```

The **goal is a function returning a reward** after each action, for example:

```python
def my_reward_function(game):
    if game.won:
        return 100
    if game.dead:
        return -100
    return -0.01  # Small penalty encourages finishing quickly
```

It does *not* understand a plain-English goal automatically. The game/developer must make success measurable.

## Build your own game

Every game needs these two methods:

```python
class MyGame:
    def reset(self):
        # Reset a new level. Return a 1D list of input numbers.
        self.player_x = 0
        self.finished = False
        return [self.player_x / 10]

    def step(self, action):
        # Called once for each action the brain chooses.
        if action == "right":
            self.player_x += 1
        elif action == "left":
            self.player_x -= 1
        self.player_x = max(0, min(10, self.player_x))
        self.finished = self.player_x == 10
        return [self.player_x / 10], self.finished, {"won": self.finished}


def goal(game):
    return 10 if game.finished else -0.01


game = MyGame()
brain = Brain(actions=["left", "right"], goal=goal)
brain.learn(game, episodes=300, max_steps=25)
print(brain.play(game, max_steps=25))
```

### Built-in ways to define a goal

Instead of writing a reward function, choose one:

```python
# Give the agent the CHANGE in score at each step:
Brain(actions=["left", "right"], goal="maximize", target="score")

# Reward every step the game remains active:
Brain(actions=["dodge", "wait"], goal="survive")

# Reward reaching a game field called finish:
Brain(actions=["left", "right"], goal="reach", target="finish")

# Reward events listed in step info (e.g. {"events": ["coin"]}):
Brain(actions=["left", "right"], goals={"coin": 5, "win": 100, "die": -50})
```

For `maximize` and `reach`, `target` must be a game attribute or a key in the `info` dictionary returned by `step()`. `maximize` requires a numeric starting target available after `reset()`. With the `events` option, emit events on the step when they occur; `info={"events": ["coin"]}` or `info={"coin": True}` works.

If your environment already returns its own reward, leave out `goal`:

```python
Brain(actions=[0, 1])  # The environment returns (obs, reward, done, info)
```

It also accepts Gymnasium-style `reset() -> (observation, info)` and `step() -> (observation, reward, terminated, truncated, info)`; you still supply an appropriate action list.

## Commands

| Method | Purpose |
|---|---|
| `Brain(actions=..., goal=...)` | Create a brain; it infers the input size from your first observation |
| `brain.think(observation)` | Choose the highest-valued action (no exploration) |
| `brain.think(observation, explore=True)` | Choose an action with epsilon-greedy exploration |
| `brain.learn(game, episodes=500, max_steps=100)` | Play training episodes and adjust neural-network weights |
| `brain.play(game, max_steps=100)` | Play a game greedily without training |
| `brain.save("filename")` | Save weights/config to `filename.npz` |
| `Brain.load("filename")` | Load a saved brain |

If your reward is a custom Python function, pass it again when **resuming training** because Python functions are intentionally not serialized:

```python
brain = Brain.load("my_brain", goal=my_reward_function)
brain.learn(game, episodes=100)
```

Loading without the callback is fine for calling `brain.think(...)` or `brain.play(...)`.

## How it learns

1. The game provides its current numeric state (e.g. player position and treasure position).
2. At the beginning, the brain often selects a random action.
3. The game responds with a new observation and any progress/failure signals.
4. The goal function computes a numeric reward.
5. The brain stores `(state, action, reward, next_state, finished)` in replay memory.
6. A feedforward neural network learns to estimate the expected future reward of each action (Q-values), using gradient descent, replay batches, and a target network.
7. Over episodes, the brain explores less and increasingly chooses actions it expects to work.

The network is built and trained using NumPy, without any teacher language model.

## Limits of this first version

- **Discrete actions only**, like `"jump"`, `"left"`, or `0` and `1`.
- **Fixed-length, numeric, 1D observations only**. A screenshot, sound clip, or raw text requires a separate encoder.
- Uses a straightforward DQN, which is not guaranteed to learn every game. Complex tasks need thoughtful rewards, more compute, and perhaps more advanced algorithms.
- The game needs a `reset()` method and a `step(action)` method. A free-form arbitrary application is not automatically controllable.
- Saving persists weights and settings, **not** the replay buffer; training can resume with fresh experience.
- Results vary by reward design, random exploration, game difficulty, and training length. Even a trained agent can make mistakes.

## Development / tests

```bash
python -m pip install -e '.[dev]'
python -m pytest -q
```

The project uses the MIT license.
