Metadata-Version: 2.4
Name: chirality
Version: 0.1.0
Summary: Piano fingering from ergonomic rules, statistical models, and reinforcement learning
Project-URL: Homepage, https://github.com/vibetuned/chirality
Project-URL: Repository, https://github.com/vibetuned/chirality
Project-URL: Issues, https://github.com/vibetuned/chirality/issues
Author-email: Pedro Fillastre <pfillastre@vibetuned.com>
License-Expression: AGPL-3.0-or-later
License-File: LICENSE
Keywords: fingering,gymnasium,hidden-markov-model,music,piano,reinforcement-learning
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.14
Requires-Dist: gymnasium>=1.2.3
Requires-Dist: matplotlib>=3.10.8
Requires-Dist: music21>=9.9.1
Requires-Dist: networkx>=3.6.1
Requires-Dist: numpy>=2.4.2
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: skrl>=1.4.3
Requires-Dist: tensorboard>=2.20.0
Requires-Dist: torch>=2.10.0
Requires-Dist: typer>=0.21.1
Description-Content-Type: text/markdown

# Chirality — an RL pipeline for piano fingering

Chirality learns and computes **piano fingerings** (which finger plays
which key) for musical scores. It models the keyboard as a lattice, encodes
ergonomic knowledge from the piano-fingering literature as cost rules,
learns statistical fingering models from human data, and offers several
solvers on top of the same representation:

- a **linear agent** — exact dynamic programming (Dijkstra over a fingering
  network), the same approach Parncutt et al. used in the original paper;
- a **VNS solver** — variable neighborhood search over whole-piece
  fingerings (after Balliauw et al.), the oracle for any rule set;
- **statistical models** — the Nakamura fingering HMMs (orders 1–3,
  Viterbi decoding) and a learned-weight rule model (path-difference
  learning), both trained on the PIG dataset of human fingerings —
  currently the systems that match human fingerings best;
- an **RL agent** — PPO (via [skrl](https://skrl.readthedocs.io/)) with
  optional Behaviour Cloning and RND exploration, trained on the
  `Piano-v0` Gymnasium environment, with the HMM log-likelihood
  available as a learned reward.

Both hands are supported: the left hand is the ergonomic mirror of the
right, handled once at the rule-aggregation boundary (see
[docs/design.md](docs/design.md)).

Headline numbers (full tables in
[docs/experiments_summary.md](docs/experiments_summary.md)): on the PIG
benchmark, human annotators agree with each other on **71.4%** of notes;
our learned-weight rule model reaches **65.4%**, the order-3 HMM
**64.4%**, the rule solvers ~59–60%, and the best PPO agent 44.8%
(right hand).

## Rule sets

The cost rules live in `src/chirality/rules/` and are validated against
the source papers:

| Module | Paper | Status |
| :--- | :--- | :--- |
| `parncutt97` | Parncutt, Sloboda, Clarke, Raekallio & Desain (1997), *An ergonomic model of keyboard fingering for melodic fragments* | All 12 rules; reproduces the paper's Table 2 and **all 88 transcribed Table 4 rows plus all 7 playable-fingering counts exactly** ([docs/table_4.md](docs/table_4.md)) |
| `jacobs01` | Jacobs (2001), *Refinements to the ergonomic model for keyboard fingering* | Physical-distance mapping (Rule A), unified large-span (Rule C), finger-4-only weak rule, rule 7 disabled |
| `balliauw17` | Balliauw, Herremans, Palhazi Cuervo & Sörensen (ITOR), *A VNS algorithm to generate piano fingerings for polyphonic sheet music* | 15 rules over a 14-key distance metric; soft practical limits; native chord rules (14, 15) |
| `alkasimi07` | Al Kasimi, Nichols & Raphael (ISMIR 2007), *A simple algorithm for automatic generation of polyphonic piano fingerings* | Vertical (within-chord) + horizontal (slice-to-slice, averaged) bathtub costs; crossover penalties |

The plan is to keep layering rule sets from further papers on the same
lattice representation. Each new rule set should follow the same pattern:
pure right-hand rule functions, mirroring at the aggregate, and a
reproduction test against the paper's own published numbers
(`tests/test_table_repro.py` is the template).

## Statistical models

The statistical branch (`src/chirality/hmm/`,
`src/chirality/rules/weight_learning.py`) is validated with the same
paper-reproduction rigor as the rule sets:

| Module | Paper | Status |
| :--- | :--- | :--- |
| `hmm` | Nakamura, Saito & Yoshii (2020), *Statistical learning and estimation of piano fingering* | Orders 1–3 with Viterbi decoding on our lattice; decoder is **note-for-note identical to the authors' reference binary** (0/10,225 mismatches, all orders); training on PIG recovers their released parameters to file precision; the paper's Table 2 match rates reproduce to 0.1 |
| `hmm.metrics` | ibid., Sec. 6.2 + App. A | All four multi-ground-truth match rates (M_gen, M_high, M_soft, M_rec); agrees with the reference evaluator to print precision; the human inter-annotator row reproduces the paper's exactly (71.4/79.1/90.8/84.3) |
| `rules.weight_learning` | Radisavljevic & Driessen (ICMC 2004), *Path difference learning for guitar fingering problem* | Structured perceptron over the 12 Parncutt rule point-sums with an exact weighted Viterbi; learned weights are the **best non-human system** on the PIG benchmark (65.4 M_gen) and interpretable (thumb-passing ×4, weak-finger rules ≈ off) |

The models double as RL machinery: the first-order HMM log-likelihood
is registered as the `hmm_nll` reward (mixable with the rule penalties),
and it is the first reward the attention policies were able to follow —
see [docs/reward_roadmap.md](docs/reward_roadmap.md).

`chirality fetch-models` downloads the pretrained artifacts from the
[Hugging Face model repo](https://huggingface.co/vibetuned/chirality-models).
To retrain from scratch instead: the training data is the
[PIG dataset](https://beam.kisarazu.ac.jp/research/PianoFingeringDataset/)
(registration required; research and non-profit use) — unpack it to
`data/PIG/`, then `chirality hmm-train` and `chirality learn-weights`
fit everything.

## Installation

```bash
pip install chirality
chirality fetch-models     # pretrained models -> ./data (add --ppo for checkpoints)
chirality annotate --input score.mei --output ./annotated --weights --both
```

`fetch-models` downloads the pretrained statistical models (learned
rule weights + fingering HMMs) from
[vibetuned/chirality-models](https://huggingface.co/vibetuned/chirality-models)
into `./data`, where every command looks by default — no account needed.

For development the project uses [uv](https://docs.astral.sh/uv/) and
Python ≥ 3.14:

```bash
uv sync
uv run pytest          # 334 tests: unit rules + paper reproductions
```

Paper-reproduction tests against the Nakamura reference artifacts and
PIG skip automatically unless `scripts/fetch_nakamura_ref.sh` has been
run and `data/PIG/` exists.

## CLI

The `chirality` command (defined in `src/chirality/cli.py`):

```bash
# Sanity-check the Gymnasium env (add --render to watch random actions)
uv run chirality verify-env [--render]

# Watch the linear (Dijkstra) agent play, with live rendering
uv run chirality linear [--rules parncutt|jacobs] [--hand 1|2] \
    [--score-type scale|chord|arpeggio|mei] [--mei-path score.mei] [--debug]

# Watch the whole-piece oracles play (VNS search / HMM Viterbi)
uv run chirality vns [--rules parncutt] [--score-type mixed]
uv run chirality hmm [--order 1|2|3] [--params param_FHMM1.txt]

# Fit the statistical models from the PIG dataset (needs data/PIG)
uv run chirality hmm-train          # HMM params -> data/hmm_params/
uv run chirality learn-weights      # path-difference rule weights

# Train the PPO agent (defaults from conf/base.yaml)
uv run chirality train [--steps 100000] [--use-bc] [--use-rnd] \
    [--score-type scale] [--resume runs/.../checkpoint.pt]

# Test a trained checkpoint with rendering
uv run chirality test --checkpoint runs/piano_ppo/.../best_agent.pt

# Annotate MEI scores with fingerings (see "Annotating scores" below)
uv run chirality annotate --input ./scores/ --output ./annotated \
    --weights --both
```

Notes:

- `--rules` selects the ergonomic model (`jacobs` is the default).
- `--hand`: 1 = right hand (treble staff), 2 = left hand (bass staff).
  `annotate --both` does both staves in one pass.
- `--debug` on `linear` saves the fingering-network graph and the env
  render for each episode under `debug_linear/`.
- `--video clip.gif` on `linear`, `vns`, `hmm`, and `test` records the
  performance to an animated GIF (3 episodes by default; `--episodes`
  overrides). `hmm` needs the reference parameters staged by
  `scripts/fetch_nakamura_ref.sh` (or `--params` for your own).
- Defaults (hand, score generator, policy architecture, BC/RND settings)
  come from [conf/base.yaml](conf/base.yaml); CLI flags override them.

## Annotating scores

`chirality annotate` writes fingering numbers (`<fing>` elements)
into MEI scores, one method per run — listed here strongest human
match first (numbers from
[docs/experiments_summary.md](docs/experiments_summary.md)):

```bash
# Learned-weight rule model (best system, 65.4% human match).
# Uses data/learned_weights/parncutt.json when present
# (from `chirality learn-weights` or the Hugging Face repo),
# uniform Parncutt weights otherwise.
uv run chirality annotate --input score.mei --output ./annotated \
    --weights --both

# Nakamura fingering HMM (orders 1-3; order 2/3 ~ 64.4% human match).
# Uses data/hmm_params/ (from `chirality hmm-train`) or the
# reference parameters staged by scripts/fetch_nakamura_ref.sh.
uv run chirality annotate --input score.mei --output ./annotated \
    --hmm --order 2 --both

# Rule-based Dijkstra agent (no trained parameters needed at all)
uv run chirality annotate --input score.mei --output ./annotated \
    --linear --rules jacobs --both

# A trained PPO checkpoint
uv run chirality annotate --input score.mei --output ./annotated \
    --model runs/.../best_agent.pt
```

`--both` fingers both staves (treble = right hand, bass = left);
`--staff 1|2` picks one. Statistical methods decode each staff as a
note sequence directly (no environment), so they are fast and handle
whole directories of `.mei` files.

## The environment and its renderer

`Piano-v0` (`src/chirality/envs/piano_env.py`) models an 88-key piano
as a **52×2 lattice**: 52 columns (one per white key) × 2 rows (row 0 =
naturals, row 1 = the sharp of that column). Hand position is derived
automatically from the pressed fingers, so the agent's action is purely a
fingering decision:

- **Action**: `fingers` (MultiBinary 5 — which finger presses) and
  `fingers_black` (MultiBinary 5 — black-key modifier per finger).
- **Observation**: the score lookahead as a `2×52×10` grid, the current
  hand position, the relative distance to the next target, and an
  `action_mask` (key colours, note count, allowed fingers).
- **Rewards**: composable `RewardComponent`s (`envs/rewards.py`), including
  the ergonomic rule penalties (`envs/parncutt_rewards.py`,
  `envs/jacobs_rewards.py`).

With `render_mode="human"` the env opens a matplotlib window with two
panels: the **score lookahead grid** (upcoming targets scrolling toward
the keyboard, with episode/step/reward in the title) and the **keyboard
view** showing the current hand span and pressed fingers.
`render_mode="rgb_array"` returns frames for capture instead.

Scores come from generators in `src/chirality/scores/`: random
`scale`s, `chord` progressions, `arpeggio`s, or real `mei` files
(`MEIScoreGenerator` parses one staff of an MEI score).

## Repository layout

```
conf/base.yaml               training / env configuration
docs/                        rule write-ups + paper reproduction notes
src/chirality/
  rules/                     ergonomic rule engine (paper-validated)
  envs/                      Piano-v0 env, reward components, renderer
  scores/                    score generators (scales, chords, MEI, demos)
  linear/                    Dijkstra fingering agent
  policies/                  PPO architectures (simple, CNN, transformer, decoder)
  exploration/               BC + RND trainers
  annotate.py                MEI fingering annotation
  cli.py                     typer CLI
tests/                       unit tests + paper reproduction tests
test_data/                   MEI corpus + annotated outputs (gitignored)
```

## Roadmap (next sessions)

- **Geometric rule module** (`rules/geometry.py`) — whole-hand costs
  computable from the lattice: hand-angle regression, frame-shift vs
  drift, grip-shape reuse, per-finger travel, hand-size-parametric
  spans ([docs/reward_roadmap.md](docs/reward_roadmap.md) §3).
- **Counterpoint generator** — reuse the VNS skeleton in
  `linear/vns.py` (Herremans & Sörensen used VNS for counterpoint
  composition): the neighbourhoods mutate *notes* instead of fingers and
  the pluggable `sequence_cost` objective becomes a counterpoint rule
  set, with fingering cost as a playability term.
- **Adversarial imitation** — GAIL-style discriminator rewards from
  VNS or PIG demonstrations; MaxEnt-IRL over the rule features as the
  principled version of the path-difference weights.
- **TiPiano** (pending release) — 340K fingertip-key contact events
  with hand motion capture: fingering supervision at scale plus
  empirical grounding for the geometry rules.
- Experiment log lives in [docs/experiments.md](docs/experiments.md)
  (abridged: [docs/experiments_summary.md](docs/experiments_summary.md));
  the training launchers append their results there automatically.

## Documentation

- [docs/design.md](docs/design.md) — architecture and technical choices
- [docs/experiments_summary.md](docs/experiments_summary.md) — the two headline result tables
- [docs/experiments.md](docs/experiments.md) — full experiment results log
- [docs/reward_roadmap.md](docs/reward_roadmap.md) — reward-model roadmap and status
- [docs/table_4.md](docs/table_4.md) — Parncutt Table 4 reproduction and findings
- [docs/table_2.md](docs/table_2.md) — combined span-rule table (rules 1–3)
- [docs/rules_paslclrade97.md](docs/rules_paslclrade97.md), [docs/rules_jacobs01.md](docs/rules_jacobs01.md) — rule write-ups

## References

Rule-based models:

- Parncutt, R., Sloboda, J. A., Clarke, E. F., Raekallio, M. & Desain, P.
  (1997). *An ergonomic model of keyboard fingering for melodic
  fragments.* Music Perception, 14(4), 341–382.
- Jacobs, J. P. (2001). *Refinements to the ergonomic model for keyboard
  fingering of Parncutt, Sloboda, Clarke, Raekallio, and Desain.* Music
  Perception, 18(4), 505–511.
- Al Kasimi, A., Nichols, E. & Raphael, C. (2007). *A simple algorithm
  for automatic generation of polyphonic piano fingerings.* ISMIR.
- Balliauw, M., Herremans, D., Palhazi Cuervo, D. & Sörensen, K. (2017).
  *A variable neighborhood search algorithm to generate piano fingerings
  for polyphonic sheet music.* International Transactions in Operational
  Research, 24(3), 509–535.

Statistical models and learning:

- Yonebayashi, Y., Kameoka, H. & Sagayama, S. (2007). *Automatic
  decision of piano fingering based on hidden Markov models.* IJCAI.
- Nakamura, E., Ono, N. & Sagayama, S. (2014). *Merged-output HMM for
  piano fingering of both hands.* ISMIR.
- Nakamura, E., Saito, Y. & Yoshii, K. (2020). *Statistical learning and
  estimation of piano fingering.* Information Sciences, 517, 68–85.
  ([arXiv:1904.10237](https://arxiv.org/abs/1904.10237); the
  [PIG dataset](https://beam.kisarazu.ac.jp/research/PianoFingeringDataset/))
- Ramoneda, P., Jeong, D., Nakamura, E., Serra, X. & Miron, M. (2022).
  *Automatic piano fingering from partially annotated scores using
  autoregressive neural networks.* ACM Multimedia.
- Radisavljevic, A. & Driessen, P. (2004). *Path difference learning for
  guitar fingering problem.* ICMC.

## License and acknowledgments

Copyright © 2026 Pedro Fillastre / [Vibetuned](https://vibetuned.com)
(non-profit). Licensed under the
[GNU AGPL v3.0 or later](LICENSE).

This project was developed in close collaboration with **Claude**
(Anthropic) — the paper reproductions, the statistical-model ports and
their validation against the reference implementations, the evaluation
harness, and large parts of the training pipeline grew out of that
pairing.

The statistical models are trained on the PIG dataset (Nakamura, Saito
& Yoshii), used under its research/non-profit terms — thanks to the
authors for releasing both the data and their reference implementation.

