Metadata-Version: 2.5
Name: terpsi
Version: 0.0.3
Summary: Dance corpora from single-dancer videos: body landmarks (no video kept), audio, subtitles, on-screen text, and movement-vs-music analysis.
Project-URL: Homepage, https://github.com/thorwhalen/terpsi
Project-URL: Repository, https://github.com/thorwhalen/terpsi
Project-URL: Documentation, https://thorwhalen.github.io/terpsi
Author: Thor Whalen
License-Expression: MIT
License-File: LICENSE
Keywords: beat,corpus,dance,landmarks,mediapipe,music,pose,thoremin
Requires-Python: >=3.10
Requires-Dist: cw
Requires-Dist: numpy
Requires-Dist: scipy
Provides-Extra: acquire
Requires-Dist: yb[download]; extra == 'acquire'
Provides-Extra: all
Requires-Dist: librosa; extra == 'all'
Requires-Dist: matplotlib; extra == 'all'
Requires-Dist: mediapipe>=0.10.30; extra == 'all'
Requires-Dist: ocracy[ocrmac]; (sys_platform == 'darwin') and extra == 'all'
Requires-Dist: ocracy[tesseract]; (sys_platform != 'darwin') and extra == 'all'
Requires-Dist: opencv-python; extra == 'all'
Requires-Dist: soundfile; extra == 'all'
Requires-Dist: yb[download]; extra == 'all'
Provides-Extra: audio
Requires-Dist: librosa; extra == 'audio'
Requires-Dist: soundfile; extra == 'audio'
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: sphinx-rtd-theme>=1.0; extra == 'docs'
Requires-Dist: sphinx>=6.0; extra == 'docs'
Provides-Extra: ocr
Requires-Dist: ocracy[ocrmac]; (sys_platform == 'darwin') and extra == 'ocr'
Requires-Dist: ocracy[tesseract]; (sys_platform != 'darwin') and extra == 'ocr'
Provides-Extra: pose
Requires-Dist: mediapipe>=0.10.30; extra == 'pose'
Requires-Dist: opencv-python; extra == 'pose'
Provides-Extra: report
Requires-Dist: matplotlib; extra == 'report'
Description-Content-Type: text/markdown

# terpsi

Dance corpora from single-dancer videos: body landmarks (no video kept), audio, subtitles, on-screen text, and movement-versus-music analysis. Named for Terpsichore, the muse of dance.

```bash
pip install "terpsi[all]"   # mediapipe + opencv, librosa, an OCR backend, yb (yt-dlp); ffmpeg on PATH
terpsi ingest https://youtu.be/q_TUyxUhoEw
terpsi clips
```

A clip goes in as a URL (YouTube, Wikimedia Commons, anything yt-dlp reads) or a local file, and comes out as a folder under `~/.local/share/terpsi/clips/<clip_id>/` holding what a dance model needs and nothing a video needs: 553 landmarks per frame (33 body + 2×21 hands + 478 face, with metric 3D world coordinates for body and hands, and 52 face blendshapes), the audio stream as the source carried it, the description and metadata, the subtitles, the text that was on screen with its timestamps, a beat grid, and a movement-versus-music report. The video is deleted once extracted.

## Python

```python
from terpsi import ingest, Corpus

clip_id = ingest("https://youtu.be/q_TUyxUhoEw")      # download, extract, delete the video
clip = Corpus()[clip_id]
lm = clip.landmarks            # LandmarkSeries: lm.pose (F,33,4), lm.pose_world (F,33,3), lm.hand_left, lm.face, ...
clip.beats["tempo"]            # 129.2
clip.screen_text[:2]           # [{'t_start': 3.0, 't_end': 6.0, 'text': 'Mise en place', ...}, ...]
clip.report["beat_alignment"]  # AIST++ Beat Alignment Score of the body's landings against the music beats
```

Each stage is a function with one seam, a keyword argument whose default works out of the box:

| stage | function | seam | default | a replacement that exists |
|---|---|---|---|---|
| fetch | `terpsi.acquire.fetch` | `downloader=` | `yb` over yt-dlp, with info.json, description, subtitles | a local file (passes through) |
| landmarks | `terpsi.pose.extract_landmarks` | `extractor=` | MediaPipe `HolisticLandmarker` | `kodokan.pose.estimate_poses` (RTMPose, 2D) |
| on-screen text | `terpsi.screen_text.read_screen_text` | `ocr=` | `ocracy` (`ocrmac` on macOS, Tesseract elsewhere) | any `ocracy` backend id |
| beats | `terpsi.audio.track_beats` | `tracker=` | librosa's beat tracker + onset envelope | a `madmom` or `su` tracker with the same signature |
| storage | `terpsi.store.Corpus` | `root=` | `~/.local/share/terpsi/clips` | any folder, a mounted or synced bucket included (a byte-store backend is the next seam, terpsi#1) |

`import terpsi` needs only numpy, scipy and cw; the heavy dependencies are extras (`pose`, `audio`, `ocr`, `acquire`) imported on first use.

## Choosing clips

```bash
terpsi search "house dance tutorial footwork" --n 15          # YouTube, Creative-Commons-filtered by default
terpsi screen https://youtu.be/XXXXXXXXXXX                      # single dancer? whole body? (a pose count on sampled frames)
terpsi ingest-many sources.txt                                  # one URL per line
```

`screen` reports the fraction of sampled frames with exactly one body, the fraction where that body is whole (nose and an ankle visible), and the licence yt-dlp found. The licence of record per clip is yt-dlp's `license` field, stored in the manifest.

## The analysis

`terpsi.analysis` implements the literature's movement-versus-music measures over a `LandmarkSeries` and a beat grid: motion energy (summed joint speed), kinematic beats (its local minima, AIST++), the Beat Alignment Score in both directions, the phase concentration of the body's landings within the beat cycle (a resultant length), the motion tempo by autocorrelation and its ratio to the music tempo, and the audio-visual lag by onset cross-correlation. `terpsi analyze <clip_id>` recomputes a clip's report.

## Data layout

```
~/.local/share/terpsi/clips/<clip_id>/
  manifest.json       id, title, source, licence, duration, fps, coverage, mirrored hint, files
  info.json           yt-dlp's metadata
  description.txt
  subtitles/*.vtt
  audio.m4a           the source audio stream (no re-encode)
  landmarks.npz/.json the LandmarkSeries (NaN where nothing was seen) and its meta
  screen_text.json    [{t_start, t_end, text, lines}]
  beats.json          tempo, beats, onset envelope
  report.json         the movement-vs-music report
```

`$TERPSI_DATA_DIR` moves the root. Models (`.task` files) are fetched once into `<root>/models/`.

## Related

[thoremin](https://github.com/thorwhalen/thoremin) (the live instrument whose body nodes and pulse follower this corpus is for), [paces](https://github.com/reeleehq/paces) (step documents from tutorial video), [kodokan](https://github.com/thorwhalen/kodokan) (pose analysis of judo video), [yb](https://github.com/reeleehq/yb), [ocracy](https://github.com/thorwhalen/ocracy). Research notes: `docs/research/`.
