Metadata-Version: 2.4
Name: gesto
Version: 0.1.1
Summary: Train and run gesture recognition models from Gesto Labeller datasets
Author: Sundar Balamurugan
License: MIT
Project-URL: Homepage, https://github.com/TheMadrasTechie/gesto
Project-URL: Issues, https://github.com/TheMadrasTechie/gesto/issues
Keywords: gesture-recognition,mediapipe,pose-estimation,computer-vision,lstm,sign-language
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy<2,>=1.23
Requires-Dist: protobuf<5,>=3.20
Requires-Dist: tensorflow<2.18,>=2.15
Requires-Dist: opencv-python<4.12,>=4.8
Requires-Dist: mediapipe<0.10.30,>=0.10
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# gesto

Train and run gesture recognition models from [Gesto Labeller](https://github.com/TheMadrasTechie) datasets.

Point it at a project folder, pick a mode and a region, and it handles the rest —
loading, training, versioned model storage, and live detection that matches how
the data was captured.

```bash
pip install gesto
```

## Two modes

| mode | the gesture is… | model | example |
|---|---|---|---|
| `static` | a held shape or posture | Dense network | thumbs up, alphabet letters, a stance |
| `sequence` | a motion over time | stacked LSTM | waving, clapping, jogging |

Static needs far less data (every captured frame is a training sample) and
predicts instantly with no warm-up. Reach for `sequence` only when two gestures
share the same shape and differ by movement.

## Five regions

| region | dim | tracks |
|---|---|---|
| `hands_one` | 63 | one hand, 21 joints |
| `hands_two` | 126 | both hands |
| `pose` | 132 | full body, 33 points |
| `legs` | 32 | lower body, 8 points |
| `full` | 258 | body + both hands |

The region must match how the project was captured — `gesto` checks the feature
dimension and tells you if it doesn't.

## Command line

```bash
# what's in this dataset?
gesto inspect ./gesto_projects/signs

# train
gesto train static hands_one ./gesto_projects/signs
gesto train sequence pose ./gesto_projects/jogging --seq-len 30

# detect (newest model by default)
gesto detect static hands_one
gesto detect sequence pose --source clip.mp4
gesto detect static hands_one --version 2

# what have I trained?
gesto list
```

## Python

```python
import gesto

run = gesto.train("./gesto_projects/signs", region="hands_one", mode="static")
gesto.detect("static", "hands_one")
```

Or drive a model yourself:

```python
from gesto.detect import Predictor

predictor = Predictor.load("static", "hands_one")
vector = predictor.features(holistic_result)   # extract + normalize
probs = predictor.predict(vector)
```

## Where models go

Everything lands under one `artifacts/` folder, split by mode then region.
Training never overwrites an earlier run — it versions:

```
artifacts/
    static/
        hands_one/          model.keras, labels.json
        hands_one_2/        the next run
        pose/
    sequence/
        pose/
        pose_2/
```

`gesto detect static pose` picks the newest version; `--version 1` picks a
specific one.

## Matching your capture

Predictions are only correct when detection feeds the model the same kind of
vector it trained on. `gesto` mirrors Gesto Labeller exactly:

- **the same engine** — MediaPipe Holistic, same confidence settings
- **the same landmark order** — including one-hand mode preferring the right
  hand and falling back to the left
- **the same normalization** — translation/scale-invariant, verified identical
- **the same mirroring** — webcam frames are flipped, video files are not

If you captured with Gesto's **Normalise** unchecked, pass `--raw` when training
so detection knows to skip it.

## Getting good results

- **Balance your classes.** Similar sample counts per class; `gesto` applies
  class weights but balanced data is better.
- **Enough samples.** ~20–30 static frames per class, or ~15–30 sequences.
  Small datasets automatically get a lighter model, since an oversized network
  on little data overfits and collapses to predicting one class.
- **Consistent clip length** for sequence mode — set "Max frames" in Gesto
  Labeller so every capture is the same length.

Run `gesto inspect <project>` to check all of this before training.

## Installation

```bash
pip install gesto
```

`gesto` uses MediaPipe's legacy `solutions` API, which was removed in MediaPipe
0.10.31. It also needs versions of TensorFlow, NumPy and protobuf that agree
with that MediaPipe — newer TensorFlow (2.21+) and OpenCV (5.0) pull protobuf
and NumPy in an incompatible direction. The package therefore pins a coherent,
tested set:

| package | pinned range | tested with |
|---|---|---|
| mediapipe | `>=0.10,<0.10.30` | 0.10.21 |
| tensorflow | `>=2.15,<2.18` | 2.17.1 |
| numpy | `>=1.23,<2` | 1.26.4 |
| protobuf | `>=3.20,<5` | 4.25.9 |
| opencv-python | `>=4.8,<4.12` | 4.11.0 |

Install into a **fresh virtual environment** so these don't clash with other
projects:

```bash
python -m venv gesto_env
# Windows:  gesto_env\Scripts\activate
# macOS/Linux:  source gesto_env/bin/activate
pip install gesto
```

If you already hit dependency conflicts (e.g. you had TensorFlow 2.21 or
OpenCV 5.0 installed), the cleanest fix is a fresh venv as above. To repair an
existing environment, pin the set explicitly:

```bash
pip install "mediapipe==0.10.21" "tensorflow==2.17.1" "numpy==1.26.4" "protobuf==4.25.9" "opencv-python==4.11.0.86"
```

## Roadmap

The legacy MediaPipe `solutions` API won't be maintained forever. A future
release will move to MediaPipe's newer **Tasks** API (`HandLandmarker`,
`PoseLandmarker`), which lifts the version ceiling. That API produces slightly
different hand-landmark geometry, so models would need retraining — hence it's a
deliberate, separate step rather than a drop-in change.

## License

MIT
