TacStack — Cross-sensor tactile semantics for robotics

Principles that hold the stack together

Four commitments, applied to every adapter, model and integration decision.

Raw-first

Keep source payloads and metadata alongside derived representations, so processing remains traceable.

Semantics-first

One Observation and Event contract spans sensors and modalities — images, taxel matrices, force/torque.

Edge-first

Core depends on neither ROS2, cloud services nor training frameworks. Runs where the robot runs.

Integration-first

Record with MCAP, inspect with Rerun and connect to existing tools. ROS2 and LeRobot integrations are planned.

From a raw signal to a useful event.

An offline workflow for developing and evaluating tactile applications, with the original data kept alongside derived outputs.

Dataset

Open-X-Tactile tar / zarr episodes; MCAP replay for recorded runs.

Observation

TactileObservation — shared metadata, sensor-specific arrays and raw payloads.

Runtime

Contact / slip baselines via builtin scorers or ONNX artifacts.

Events

TactileEvents marked on a synchronized Rerun timeline, annotatable.

Benchmark

Reproducible per-episode reports with quality and calibration tracking.

Software ready. Hardware is next.

Four protocol codecs have tests based on vendor manual examples. Live acquisition and real-world stability will be verified when the sensors arrive.

Sensor	Output	Link	Status
M0404Sresistive matrix kit	4×4 matrix · 16 taxels	UART 115200, active push	● codec ready
PaXini PX-6AX GEN3tactile skin	per-point 3-axis force + resultant	UART 921600, req–resp	● codec ready
PaXini PX6Dsix-axis F/T	Fx, Fy, Fz, Mx, My, Mz · float32	USB / RS485 · 1 kHz	● codec ready
PaXini PX3Qjoint torque	Mx, My, Mz · N·m · 30/50/100 FS	USB / RS485 · 921600	● codec ready

PX6D / PX3Q response CRC validation remains open until hardware captures are available. Protocol tests do not establish live sensor performance. Hardware testing pending

Where the project stands

Phase 0–4 are complete. The next milestone is the first live sensor.

PHASE 0–2 · DONE

Contracts, data path, replay

Package and core contracts; Open-X-Tactile adapter for image / matrix / F/T streams; MCAP export and Rerun replay with C/S/U annotation.

PHASE 3 · DONE

Runtime & benchmark

Builtin and ONNX scoring backends, temporal contact / slip baselines and repeatable reports. Model-quality evaluation requires labeled data.

PHASE 4 · DONE

Quality & calibration

Two-level capability validation, dataset quality health reports, calibration provenance, per-stream benchmark grouping.

PHASE 5 · IN PROGRESS

First live sensor

Live serial adapter, live record / replay / inference, device reconnect handling and a 30-minute stability run. Codecs are ready; hardware is on the way.

PHASE 6 · PLANNED

v0.1 release

External-user Quickstart, English documentation and the first tagged release.

Start with a working offline demo.

Python 3.12+ and uv. Install from source, then explore the bundled synthetic fixture. No sensor or dataset download needed.

The bundled fixture and the event below are synthetic. Use your own recordings to evaluate model behavior. Not yet published on PyPI.

terminal

# clone and install (uv manages everything)
git clone https://github.com/mailes/TacStack.git
cd TacStack
uv sync --extra rerun

# guided tour — no data download, no hardware
uv run tacstack demo

# replay the bundled synthetic fixture in Rerun
uv run tacstack replay \
  tests/fixtures/open_x_tactile/demo_wipe.tar --viewer

contract_example.py

from tacstack import TactileEvent

sample = TactileEvent(
    timestamp_ns=0,
    sensor_id="synthetic",
    kind="contact_begin",
    probability=1.0,
    model_id="contract-example",
    latency_ms=0.0,
)