Metadata-Version: 2.4
Name: tlabel
Version: 0.2.0b2
Summary: TouchLabel AI - Tactile Data Annotation Toolkit
Author-email: Niuzu Tech <luoxi@touchlabelai.cn>
License: MIT
Project-URL: Homepage, https://github.com/liesliy/tlabel
Keywords: tactile,annotation,robotics,GelSight,PaXini,touch
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20
Provides-Extra: gelsight
Requires-Dist: opencv-python>=4.0; extra == "gelsight"
Provides-Extra: paxini
Requires-Dist: h5py>=3.0; extra == "paxini"
Provides-Extra: daimon
Requires-Dist: pyarrow>=10.0; extra == "daimon"
Requires-Dist: opencv-python>=4.0; extra == "daimon"
Requires-Dist: scipy>=1.7; extra == "daimon"
Provides-Extra: all
Requires-Dist: opencv-python>=4.0; extra == "all"
Requires-Dist: h5py>=3.0; extra == "all"
Requires-Dist: pyarrow>=10.0; extra == "all"
Requires-Dist: scipy>=1.7; extra == "all"
Requires-Dist: pillow>=9.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: jupyter; extra == "dev"
Dynamic: license-file

# TouchLabel AI 🦞

<h3 align="center">Sensor-Agnostic Tactile Data Annotation Toolkit</h3>
<p align="center"><strong>load → review → export · Three steps to close the loop</strong></p>

<p align="center">
  <a href="https://pypi.org/project/tlabel/"><img src="https://img.shields.io/pypi/v/tlabel?color=e85d75" alt="PyPI"></a>
  <a href="https://pypi.org/project/tlabel/"><img src="https://img.shields.io/pypi/pyversions/tlabel" alt="Python"></a>
  <a href="https://github.com/liesliy/tlabel/blob/main/LICENSE"><img src="https://img.shields.io/pypi/l/tlabel" alt="License"></a>
  <a href="https://github.com/liesliy/tlabel/stargazers"><img src="https://img.shields.io/github/stars/liesliy/tlabel?style=social" alt="GitHub Stars"></a>
  <a href="https://pepy.tech/projects/tlabel"><img src="https://img.shields.io/pepy/dt/tlabel?color=blue" alt="Downloads"></a>
  <a href="README_CN.md">中文文档</a>
</p>

---

## 🔥 What's New in v0.2.0b1

**Major improvements for downstream compatibility and user experience:**

- ✨ **Enhanced metadata**: Added `sensor_id`, `calibration_params`, `feature_names`, and episode boundary markers (`is_first`/`is_last`)
- 🔄 **LeRobot integration**: Bidirectional converters for seamless data exchange with Hugging Face LeRobot framework
- 💾 **HDF5 export**: Scientific computing standard format for research workflows
- 📚 **Comprehensive tutorials**: Step-by-step guides for GelSight, PaXini, and Daimon sensors
- 🎯 **Better error messages**: Clear guidance with exact `pip install` commands and format descriptions
- 📖 **Improved docs**: 5-Minute Quick Start, sensor-specific loading instructions, troubleshooting table

See [CHANGELOG.md](CHANGELOG.md) for full details.

---

> 🎯 **Got tactile data from different sensors that refuse to talk to each other?**
> TLabel makes them speak the same language — load any format, annotate in a visual panel, export a unified schema.

---

## 🎯 5-Minute Quick Start

New to TLabel? Start here. This tutorial gets you from zero to annotated data in 5 minutes — no sensor data needed.

### Step 1: Install (30 seconds)

```bash
pip install tlabel
```

That's it. The core package installs in seconds with just numpy as a dependency.

### Step 2: Try the Built-in Demo (1 minute)

Open a Python terminal or Jupyter notebook:

```python
import tlabel

# Load built-in GelSight demo data (no files needed!)
data = tlabel.demo()

# Open the interactive annotation panel
data.review()
```

A colorful panel pops up right in your notebook:
- **Timeline** at the top shows contact (green), slip (red), and idle (gray) frames
- **Radar chart** displays all 22 dimensions for the current frame
- **Detail panel** lets you inspect and edit individual values

![TLabel Panel with Export Buttons](docs/demo_panel_with_export_zh.png)

**Try this:** Click on different frames in the timeline. Notice how the radar chart updates. Spot any frames that look wrong (e.g., `contact=0` but `force_magnitude > 0`).

### Step 3: Make Your First Correction (2 minutes)

Found a mislabeled frame? Fix it:

```python
# In the panel, use the batch correction tool:
# 1. Select a range of frames (drag on timeline)
# 2. Set contact=0 in the correction panel
# 3. Click "Apply" — cascade rules auto-zero 7 related fields

# Or do it programmatically:
data.batch_patch(10, 20, "contact", 0)  # Frames 10-20: no contact
```

The cascade system ensures physical consistency: setting `contact=0` automatically zeroes `force_magnitude`, `slip_event`, and resets `manipulation_phase` to `"idle"`.

### Step 4: Export Your Annotations (30 seconds)

```python
# Export to JSON (full TLabel Format v2 schema)
data.export("my_annotations.json")

# Or export to CSV for quick analysis in Excel/pandas
data.export("my_annotations.csv")
```

**Done!** You've just completed the full annotation loop: load → review → correct → export.

### What's Next?

- **Have real sensor data?** See the step-by-step tutorials:
  - [GelSight / DIGIT Tutorial](docs/tutorial-gelsight.md)
  - [PaXini PXCap Tutorial](docs/tutorial-paxini.md)
  - [Daimon DM-TacClaw Tutorial](docs/tutorial-daimon.md)
- **Want to understand the 22 dimensions?** Check out [TLabel Format v2](#tlabel-format-v2--22-dimensions)
- **Need a web-based tool for your team?** Try [tlabel-web](https://github.com/liesliy/tlabel/tree/main/tlabel-web)
- **Try the interactive demo in your browser:** [Live Demo](https://liesliy.github.io/tlabel/demo.html)

---

## 📥 Loading Your Own Data

Once you're comfortable with the demo, load your real sensor data:

### GelSight / DIGIT (vision-based tactile sensors)

```python
# Install extra dependencies
pip install tlabel[gelsight]

# Load your .pkl file (from gelsight-force-estimation or similar)
data = tlabel.load("my_gelsight_episode.pkl")
data.review()
```

**What happens:** The adapter extracts 22 dimensions from background-subtracted tactile images, including force magnitude, slip detection, optical flow, and manipulation phase inference.

### PaXini PXCap (distributed force array)

```python
pip install tlabel[paxini]

# Load your .h5 or .hdf5 file
data = tlabel.load("my_paxini_episode.h5")
data.review()
```

**What happens:** The adapter reads 6D force/torque vectors from each taxel region, detects contact and slip events, and maps them to 20 TLabel dimensions (no optical flow for force-only sensors).

### Daimon DM-TacClaw (multimodal robot arm + tactile)

```python
pip install tlabel[daimon]

# Load from LeRobot-style directory structure
data = tlabel.load("path/to/daimon_episode/")
data.review()
```

**Requirements:** The directory should contain:
- `meta/info.json` — episode metadata
- `data/chunk-*.parquet` — observation data (114-dim state vector)
- `videos/` — FFV1-encoded tactile video files (optional, degrades gracefully if missing)

**What happens:** The adapter decodes FFV1 video frames, extracts tactile features (deformation, contact area, texture), and fuses them with robot state data.

### Troubleshooting Common Issues

| Error | Solution |
|-------|----------|
| `ImportError: No module named 'cv2'` | Run `pip install tlabel[gelsight]` or `pip install opencv-python` |
| `ImportError: No module named 'h5py'` | Run `pip install tlabel[paxini]` or `pip install h5py` |
| `ImportError: No module named 'pyarrow'` | Run `pip install tlabel[daimon]` or `pip install pyarrow` |
| `ValueError: Unknown format` | Check file extension (.pkl, .h5, .hdf5, .parquet) or pass `format="gelsight"` explicitly |
| `FileNotFoundError` | Double-check the path; use absolute paths if unsure |

---

## ⚡ 30-Second Demo

**👉 [Try it live in your browser](https://www.coze.cn/s/f-KJdzphlHs/)** — no install needed, see the panel in action right now.

Or fire it up in Jupyter:

```python
import tlabel
data = tlabel.demo()    # ← built-in GelSight demo
data.review()           # ← interactive panel pops up
```

Or try other sensors:

```python
tlabel.demo('digit').review()    # DIGIT sensor
tlabel.demo('paxini').review()   # PaXini force sensor
tlabel.demo('daimon').review()   # Daimon DM-TacClaw
```

**What you'll see:** a color-coded timeline (🟢 contact / 🔴 slip / ⬜ idle), 22-dim radar chart, frame detail editor, and batch patching — all in one panel.

![TLabel Panel Demo](docs/demo_panel_zh.png)

---

## 🚀 Quick Start

```bash
pip install tlabel
```

```python
import tlabel

# Load — auto-detect sensor format, no config needed
data = tlabel.load("gelsight_force.pkl")     # GelSight / DIGIT
data = tlabel.load("paxini_episode.h5")      # PaXini
data = tlabel.load("daimon_data/")           # Daimon (directory or .parquet)

# Annotate — interactive Jupyter panel
data.review()          # Chinese UI
data.review(lang="en") # English UI

# Export — unified TLabel Format
data.export("output.json")   # TLabel Format v2 JSON
data.export("output.csv")    # flat CSV
```

# Pre-annotate — AI-assisted, review & correct
from tlabel.predict import PredictEngine
engine = PredictEngine()
results = engine.predict(data)          # predict contact, slip, phase...
engine.apply(data, results, min_confidence=0.7)  # apply only high-confidence
data.review()                           # review & correct in panel
```

That's it. Full loop. 🔁

---

## 🤔 Why TLabel?

| Pain | TLabel's Answer |
|------|-----------------|
| Every sensor spits out a different format | **One `load()` call, auto-detected** |
| Raw tactile data is unreadable numbers | **Visual panel: timeline + radar + details** |
| Fixing labels frame-by-frame is soul-crushing | **Batch patch + cascade rules, fix ranges in one click** |
| Your lab mate exported... something... | **Unified TLabel Format v2, 22 dimensions, no ambiguity** |
| "But we use DIGIT and they use PaXini" | **Sensor-agnostic. Load both, same schema, same tool.** |
| "Manually labeling thousands of frames is tedious" | **AI pre-annotation: predict contact, slip, phase — review & correct** |

---

## 📡 Supported Sensors

| Sensor | Format | Dimensions | Optical Flow | Status |
|--------|--------|:----------:|:------------:|:------:|
| **GelSight Mini** | `.pkl` | 22 | ✅ | ✅ Stable |
| **DIGIT** | `.pkl` | 22 | ✅ | ✅ Stable |
| **Daimon DM-TacClaw** | `.parquet` / dir | 22 (video) / 20 (no video) | ✅ / — | ✅ Stable |
| **PaXini PXCap** | `.h5` / `.hdf5` | 20 | — | ✅ Stable |

> Force-type sensors (PaXini) lack optical images → 20 dims. Image-type → full 22. Daimon gracefully degrades when no video is present. No errors, no surprises.

---

## 📦 Installation

```bash
# Just the core (numpy only, ~2s install)
pip install tlabel

# Per-sensor extras
pip install tlabel[gelsight]   # GelSight / DIGIT → opencv-python
pip install tlabel[paxini]     # PaXini → h5py
pip install tlabel[daimon]     # Daimon → pyarrow + opencv-python

# I want it all
pip install tlabel[all]
```

---

## 🎨 Panel Features

- 🎨 **Color-coded timeline**: green = contact · red = slip · gray = idle — patterns jump out instantly
- 🕸 **22-dim radar chart**: see the full feature vector at a glance, bilingual labels
- ✏️ **Frame & batch patching**: fix one frame or a range, your call
- 🔗 **Cascade rules**: set `contact=0` → 7 related fields auto-zero + phase resets to `idle`
- 🌐 **Bilingual toggle**: 中文 / English, one click top-right
- 📤 **Export**: JSON / CSV, auto-detected by file extension

---

## TLabel Format v2 — 22 Dimensions

### Static Features (18-dim)

| # | Key | Description |
|---|-----|-------------|
| 1 | `contact` | Binary contact flag |
| 2 | `deformation_magnitude` | Surface deformation intensity |
| 3 | `force_magnitude` | Normal force magnitude |
| 4 | `force_peak` | Peak force in episode window |
| 5 | `force_direction` | Force vector angle (°) |
| 6 | `slip_entropy` | Uncertainty of slip detection |
| 7 | `slip_event` | Binary slip event flag |
| 8 | `texture_energy` | Surface texture frequency energy |
| 9 | `edge_density` | Contact edge pixel ratio |
| 10 | `contact_area` | Contact region area ratio |
| 11 | `centroid_x` | Contact centroid x-position |
| 12 | `normal_field_magnitude` | Normal pressure field magnitude |
| 13 | `normal_field_variance` | Normal field spatial variance |
| 14 | `shear_field_magnitude` | Shear stress magnitude |
| 15 | `shear_field_direction` | Shear direction angle (°) |
| 16 | `delta_force_normal` | Frame-to-frame ΔF_normal |
| 17 | `delta_force_shear` | Frame-to-frame ΔF_shear |
| 18 | `friction_cone_ratio` | Tangential/normal force ratio |

### Temporal Features (4-dim, v0.2.0)

| # | Key | Image-type | Force-type | Description |
|---|-----|:----------:|:----------:|-------------|
| 19 | `optical_flow_magnitude` | ✅ | — | Inter-frame motion magnitude (Farneback) |
| 20 | `optical_flow_direction` | ✅ | — | Optical flow angle (°) |
| 21 | `temporal_deformation_rate` | ✅ | ✅ | Rate of deformation change |
| 22 | `contact_transition` | ✅ | ✅ | Contact state transition probability |

---

## 📖 API Quick Reference

```python
import tlabel

# ── Loading ──
data = tlabel.load(path)                     # Auto-detect sensor format
data = tlabel.load(path, format="gelsight")  # Force specific adapter

# ── Demo ──
data = tlabel.demo()                         # Built-in demo data
tlabel.list_demos()                          # See available sensors

# ── Properties ──
data.num_frames        # int — total frame count
data.duration_s        # float — episode duration
data.sensor_type       # str — sensor identifier
data.dimension_keys    # list — all dimension keys
data.modified_count    # int — frames with manual patches

# ── Frame Access ──
frame = data[0]                          # Index access
frame = data.get_frame(42)               # By frame_idx
frame.contact                            # Contact value
frame.slip_event                         # Slip event value
frame.is_modified                        # Has patches?

# ── Patching ──
frame.patch("contact", 0)                         # Single frame (cascade=True)
frame.patch("contact", 0, cascade=False)           # No cascade
data.batch_patch(10, 50, "contact", 0)             # Range patch

# ── Review & Export ──
data.review()                    # Jupyter panel (Chinese)
data.review(lang="en")           # English
data.export("output.json")       # JSON (TLabel Format v2)
data.export("output.csv")        # CSV
```

### Cascade Rules (contact → 0)

When `contact` is set to `0`, these fields are automatically zeroed:

| Auto-zeroed Field | Condition |
|-------------------|-----------|
| `force_magnitude` | always |
| `force_peak` | always |
| `slip_event` | always |
| `delta_force_normal` | always |
| `delta_force_shear` | always |
| `contact_area` | always |
| `contact_transition` | only if value > 0.5 |
| `manipulation_phase` | → `"idle"` (if not already) |

### 🤖 AI-Assisted Pre-Annotation

Let the engine suggest labels, then you review and correct — **human-in-the-loop, not black-box**.

```python
from tlabel.predict import PredictEngine

engine = PredictEngine()

# Option 1: cold start — no prior labels
results = engine.predict(data)

# Option 2: warm start — learn from partially labeled data first
engine.fit(data)        # extract statistics from existing annotations
results = engine.predict(data)

# Apply only high-confidence predictions (≥0.7)
applied = engine.apply(data, results, min_confidence=0.7)
print(f"Auto-filled {applied} fields")

# Review the summary
stats = engine.summary(results)
print(stats)
# → {total_frames, predicted_fields, avg_confidence, method_distribution, ...}

data.review()  # check predictions in the panel, correct as needed
```

**What it predicts:**

| Dimension | Method | Confidence Range |
|-----------|--------|:----------------:|
| `contact` | Rule (force + deformation + area) | 0.4 – 0.9 |
| `slip_event` | Rule (shear + delta + entropy) | 0.55 – 0.8 |
| `manipulation_phase` | Rule (contact + slip + force cascade) | 0.55 – 0.65 |
| Missing dims (with `fit()`) | Statistics (mean from labeled frames) | ~0.4 |

> 💡 **Tip**: Use `fit()` on partially labeled data first — even 10–20% labeled frames significantly improve statistical predictions. Predictions below your confidence threshold are simply skipped, so you stay in control.

---

## 🗂 Project Structure

```
tlabel/
├── core/
│   ├── types.py          # TLabelFrame / TLabelData containers
│   ├── loader.py         # Auto-detect & dispatch loading
│   └── registry.py       # Adapter registry
├── adapters/
│   ├── base.py           # BaseAdapter interface
│   ├── gelsight.py       # GelSight Mini / DIGIT
│   ├── paxini.py         # PaXini PXCap
│   └── daimon.py         # Daimon DM-TacClaw (+ video decoding)
├── viewer/
│   ├── panel.py          # Jupyter _repr_html_ renderer
│   └── templates.py      # HTML + JS + CSS template engine
├── predict/
│   └── engine.py         # AI-assisted pre-annotation engine
├── demo.py               # Built-in demo data loader
└── export/
    └── writer.py         # JSON / CSV export + NumpyEncoder
```

---

## 💬 Feedback

Found a bug? Have an idea? Just want to say hi?

- 🐛 **Bug report** → [Open an Issue](https://github.com/liesliy/tlabel/issues)
- 💡 **Feature request** → [GitHub Discussions](https://github.com/liesliy/tlabel/discussions)
- 🌟 **Using tlabel in your research?** → We'd love to hear about it! Drop us a star ⭐

## 🤝 Contributing

We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

**Good first issues:**
- 🔌 Add a new sensor adapter (SynTouch? XELA? Your call.)
- 📊 Improve radar chart UI (dark mode, interactive hover)
- 🌐 Add more language support (日本語, 한국어)
- 🧪 Add integration tests for edge cases

---

## 📄 License

[MIT](LICENSE) © Niuzu Tech

---

<p align="center">
  <strong>If this saved you from manually labeling tactile data, a ⭐ would make our day!</strong>
</p>
