Metadata-Version: 2.4
Name: trackclean
Version: 0.1.0
Summary: Clean, smooth, segment and measure raw GPS trajectories: GPX/TCX/GeoJSON/CSV I/O, outlier and drift removal, Kalman/RTS smoothing, stay points, trips and honest metrics.
Author-email: Yusuf Eminoğlu <yusufeminoglu@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/YusufEminoglu/trackclean_sdk
Project-URL: Repository, https://github.com/YusufEminoglu/trackclean_sdk
Project-URL: Issues, https://github.com/YusufEminoglu/trackclean_sdk/issues
Project-URL: Changelog, https://github.com/YusufEminoglu/trackclean_sdk/blob/main/CHANGELOG.md
Keywords: gps,gpx,tcx,geojson,trajectory,track,mobility,cleaning,outlier-detection,kalman-filter,rts-smoother,hampel-filter,stay-points,trip-segmentation,douglas-peucker,elevation-gain,telematics,fitness,cycling,running
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20.0
Requires-Dist: tzdata; sys_platform == "win32"
Provides-Extra: pandas
Requires-Dist: pandas>=1.3.0; extra == "pandas"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: pandas>=1.3.0; extra == "dev"
Dynamic: license-file

# trackclean

**Clean, smooth, segment and measure raw GPS trajectories in pure Python + NumPy.**

[![CI](https://github.com/YusufEminoglu/trackclean_sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/YusufEminoglu/trackclean_sdk/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/trackclean.svg)](https://pypi.org/project/trackclean/)
[![Python](https://img.shields.io/pypi/pyversions/trackclean.svg)](https://pypi.org/project/trackclean/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

Raw fixes from phones, sports watches, loggers and fleet telematics are messy. They contain
spikes hundreds of metres off, teleports after tunnels, duplicated and out-of-order
timestamps, drift that adds kilometres while you stand still, and elevation noise that turns
a 130 m hike into "13 000 m of ascent". `trackclean` is written for mobility researchers,
sports and fitness developers and transport analysts. It reads the common formats, removes
these errors with documented algorithms and reports distance, moving time, gain and stops
that hold up against ground truth.

- **Formats:** GPX 1.0/1.1 (tracks, segments, waypoints, routes; heart rate, cadence,
  power, temperature and other extensions are preserved), TCX, GeoJSON (LineString or
  Points) and CSV with automatic column detection.
- **Cleaning:** time parsing, sorting and deduplication with time-zone handling;
  speed and acceleration outlier removal tuned per travel mode; teleport detection;
  collapse of standstill drift; Hampel filter; Kalman/RTS smoother that uses per-fix
  accuracy when present; elevation smoothing and gain/loss with a hysteresis threshold;
  resampling to fixed time or distance steps; simplification (Douglas-Peucker,
  Visvalingam-Whyatt, time-aware TD-TR).
- **Segmentation:** stay points (Li et al. 2008), trip and stop segmentation, splitting at
  signal gaps, and a transport-mode guess that is *clearly labelled as a heuristic*.
- **Metrics:** haversine and Vincenty distance, moving vs elapsed time, pace, speed
  profile, elevation gain/loss, radius of gyration and a stops summary.
- **Privacy:** trim the first and last N metres around home or work.
- **CLI:** `trackclean clean | stats | segment | convert`.

The only dependency is NumPy (plus `tzdata` on Windows). pandas is optional.

## Install

```bash
pip install trackclean            # core
pip install "trackclean[pandas]"  # + DataFrame import/export
```

## Before and after: synthetic ground truth

The table below compares raw and cleaned tracks against their ground truth. Every track is
synthetic, generated from a seed by `trackclean.synthetic.make_synthetic`, so no real
person's location data is used. Each track has three moving legs separated by two
25-minute stops, sampled at 1 Hz (about 5 600 fixes). The generator then adds:

- 4 m white noise and 2.5 m slowly varying (correlated) error per axis;
- 7 m wandering indoor drift while stopped;
- 1 % spikes of 100 to 600 m and one 6-fix multipath burst of about 400 m;
- one 2-minute signal gap;
- 1 % duplicated fixes and 5 out-of-order fixes;
- elevation noise of about 4 m, with occasional 40 m spikes.

"Raw" means the fixes were only sorted and deduplicated, so that times align with the
truth. Each value is the mean over 5 seeds
(`python examples/benchmark_synthetic.py`; the same checks run in `tests/test_accuracy.py`).

| metric | walk: raw → clean (truth) | bike: raw → clean (truth) | car: raw → clean (truth) |
|---|---|---|---|
| position RMSE, m | 41.6 → **3.3** | 41.6 → **3.0** | 41.6 → **3.0** |
| median error, m | 7.5 → **1.9** | 7.5 → **1.9** | 7.5 → **1.9** |
| max error, m | 595 → **30** | 595 → **24** | 595 → **26** |
| distance, km | 80.98 → **3.68** (3.77) | 85.05 → **14.23** (14.75) | 99.91 → **34.81** (34.87) |
| elevation gain, m | 13 130 → **127** (133) | 13 113 → **449** (443) | 13 195 → **1 102** (1 066) |
| moving time, min | 92.8 → **42.5** (44.9) | 91.8 → **42.9** (44.9) | 91.2 → **43.1** (44.9) |
| stay-point recall | 0.00 → **1.00** | 0.00 → **1.00** | 0.00 → **1.00** |

Cleaning takes about 0.1 s for 5 600 fixes. The cleaned distance is slightly short
because the 2-minute signal gap is not bridged: steps between segments are not counted
unless you ask for them with `across_segments=True`. On the raw track, the spikes break
every 25-minute stop into pieces shorter than the 20-minute threshold, so Li's detector
finds none of them.

## Quick start

```python
import trackclean as tc

track = tc.read_track("ride.gpx")            # GPX / TCX / GeoJSON / CSV
result = tc.clean(track, mode="bike")        # walk, run, bike, car, transit
print(result.report)                         # what every step removed or changed
stats = tc.summarize(result.track, mode="bike")
print(stats.format())

seg = tc.segment_trips(result.track)         # stops (Li et al. 2008) + trips between them
for trip in seg.trips:
    print(trip.start_time, len(trip.track), trip.mode.mode)   # heuristic mode guess

safe = tc.trim_ends(result.track, 300, randomize=0.5)   # hide home/work
tc.write(safe, "ride.clean.gpx")             # extensions (hr, cad, power...) kept
```

### Command line

```bash
trackclean clean in.gpx -o out.gpx --mode bike
trackclean clean phone.csv -o out.geojson --mode walk --resample 5s --simplify 3 --trim-ends 300
trackclean stats in.gpx --mode run            # add --json for machine-readable output
trackclean stats in.gpx --clean --mode bike   # clean first, then measure
trackclean segment in.csv -o trips.geojson --mode car --stay-distance 200 --stay-time 1200
trackclean convert in.tcx -o out.gpx
```

Use `--tz Europe/Istanbul` when the input has timestamps without an offset.
Use `--local-time TZ` when a logger wrote local clock time but marked it as UTC.

## How it works

`tc.clean(track, mode)` runs these steps in order. Each step can be switched off, and
each is also available as its own function.

1. **Validity, order, duplicates** (`remove_invalid`, `sort_and_dedup`): drops fixes
   with NaN or out-of-range coordinates and exact `0, 0` "null island" fixes. Sorts by
   time and removes repeated timestamps, keeping either the first fix or the one with the
   smallest `accuracy`.
2. **Signal gaps** (`split_segments_on_gaps`): starts a new segment when the time step is
   longer than the mode's `max_gap`. Smoothing and distance never bridge a segment break.
3. **Speed/acceleration outliers and teleports** (`remove_outliers`, `detect_jumps`):
   a step is impossible when it is longer than
   `min(max_speed, v_ref + max_accel·dt)·dt + k·σ_pair`. Here `v_ref` is a robust local
   median speed and `σ_pair` is the combined noise of the two fixes, which keeps normal
   scatter on slow modes from being flagged. If the track jumps away and jumps back
   within 30 fixes and 5 minutes, and the bridging step is plausible, the fixes in
   between are removed (single spikes and multipath bursts). A jump that never returns
   is a real discontinuity, such as reacquisition after a tunnel, so the track is split
   there instead.
4. **Hampel filter** (`hampel_filter`): applied to east and north metres. It replaces
   moderate outliers that pass the kinematic gate with the local median.
5. **Kalman filter + Rauch-Tung-Striebel smoother** (`kalman_smooth`, `kalman_rts`):
   uses a constant-velocity model with white-acceleration process noise `q`, set by the
   mode. Measurement noise is taken per fix from `accuracy` (phones report it as a 68 %
   radius, so σ = accuracy / 1.515), otherwise from `hdop × 4 m`, otherwise from the
   mode default. The covariance recursion runs once per segment, which keeps 10 000
   fixes well under a second.
6. **Standstill drift** (`collapse_stationary`): finds stays (for example 25 m and
   2 min for walking) and moves every fix of a stay to its median centre. Adjacent
   pieces of the same stop are merged first. Timestamps are kept, so dwell time
   survives while the phantom "jitter distance" disappears.
7. **Elevation** (`smooth_elevation`): a Hampel filter removes spikes, then values are
   averaged over a 100 m window of path length. Smoothing by distance instead of by
   sample treats slow and fast travel alike. It also averages all the fixes taken while
   standing still, which is where most phantom ascent comes from.

**Elevation gain/loss** (`elevation_gain_loss(ele, threshold=5)`) reduces the profile to
its peaks and valleys. A turn only counts once the elevation has moved `threshold`
metres back from the running extreme. This is the dead-band approach that sports devices
use to report realistic ascent. `threshold=0` gives the naive sum.

**Simplification** (`simplify(track, tol, method)`) offers three methods:

- `dp`: Ramer-Douglas-Peucker, with the perpendicular distance in metres.
- `vw`: Visvalingam-Whyatt, where the tolerance is an effective triangle area in m² and
  areas are kept monotone.
- `tdtr`: top-down time-ratio (Meratnia and de By 2004). It uses the *synchronized
  Euclidean distance*, so stops and speed changes survive.

**Stay points** (`detect_stay_points(track, distance=200, min_duration=1200)`) follow
Li et al. (2008). From an anchor fix, the scan continues while the following fixes stay
within `distance`. If they span at least `min_duration`, they form a stay point. A
signal gap inside a stay (no fixes indoors) is handled naturally. The centre is the
median by default; use `centroid="mean"` for the paper's mean.

**Mode inference** (`infer_mode`) is a **heuristic, not a classifier**. It applies rules
to the median, 85th and 95th percentile speeds and the 95th percentile of acceleration,
and returns `stationary`, `walk`, `bike`, `motorized` or `high_speed_rail_or_air`, with
the features it used. Expect slow cycling and running to be confused, and car, bus and
tram to be indistinguishable.

**Radius of gyration** is the RMS distance of all fixes from their centre of mass
(González, Hidalgo and Barabási 2008).

### Mode profiles

| mode | max speed | max accel | GPS σ | Kalman q | drift radius / time | gap split |
|---|---|---|---|---|---|---|
| walk | 4 m/s | 3 m/s² | 6 m | 0.05 | 25 m / 120 s | 120 s |
| run | 8 m/s | 4 m/s² | 6 m | 0.1 | 25 m / 90 s | 120 s |
| bike | 22 m/s | 4 m/s² | 6 m | 0.1 | 25 m / 90 s | 120 s |
| car | 70 m/s | 8 m/s² | 8 m | 0.3 | 30 m / 90 s | 180 s |
| transit | 90 m/s | 4 m/s² | 10 m | 0.2 | 40 m / 60 s | 300 s |

To make your own profile, change a preset: `tc.MODES["bike"].with_(max_speed=30)`.

## Data model

A `Track` holds parallel NumPy arrays: `lat`, `lon`, `time` (epoch seconds, UTC; `nan`
when unknown), `ele`, `segment` (integer ids) and `fields`. `fields` is a dict of extra
per-point channels such as `hr`, `cad`, `power`, `speed`, `accuracy` and `hdop`. Every
step carries the fields along, and every writer writes them back. A `TrackSet` is what a
file holds: tracks, waypoints and metadata. `tc.read()` returns a `TrackSet`, and
`tc.read_track()` merges it into one `Track`. `Track.to_dataframe()` and
`Track.from_dataframe()` need the `pandas` extra.

| format | read | write | notes |
|---|---|---|---|
| GPX 1.1 / 1.0 | ✓ | ✓ | Garmin TrackPointExtension (hr, cad, atemp, ...), `<power>`, hdop/vdop/pdop/sat; other numeric fields go in a `trackclean:` namespace |
| TCX | ✓ | ✓ | activities and courses, laps as segments, HR, cadence, ActivityExtension speed/watts |
| GeoJSON | ✓ | ✓ | LineString with `coordTimes`/`coordinateProperties` (togeojson convention) or Point features |
| CSV/TSV | ✓ | ✓ | delimiter sniffing, decimal comma, ISO or epoch s/ms/µs times, separate date+time columns, `track_id` splitting, km/h speeds |
| FIT | - | - | not supported; convert with GPSBabel or the vendor app |

Time zones: naive timestamps are read in `tz=` (an IANA name or `+hh:mm`; the default is
UTC). `shift_naive_time(track, tz)` fixes tracks whose local clock time was written as
UTC. DST transitions are handled.

## Privacy

`trim_ends(track, 300)` removes **every** fix within 300 m of the first and last fix,
including fixes where the track passes home again later. A fixed radius still lets
someone guess the centre of the circle, so `randomize=0.5` enlarges each radius by a
random factor between 0 and 50 %. Treat any trajectory as personal data, and keep raw
tracks out of public repositories. This project's tests use synthetic tracks only.

## Development

```bash
pip install -e ".[dev]"
pytest --cov=trackclean
ruff check . && ruff format --check src tests && mypy src
python examples/benchmark_synthetic.py
```

## References

- Li, Q., Zheng, Y., Xie, X., Chen, Y., Liu, W., Ma, W.-Y. (2008). Mining user similarity
  based on location history. *ACM SIGSPATIAL GIS '08*.
- Vincenty, T. (1975). Direct and inverse solutions of geodesics on the ellipsoid with
  application of nested equations. *Survey Review* 23(176).
- Rauch, H. E., Tung, F., Striebel, C. T. (1965). Maximum likelihood estimates of linear
  dynamic systems. *AIAA Journal* 3(8).
- Pearson, R. K. (2002). Outliers in process modeling and identification (Hampel
  identifier). *IEEE Trans. Control Systems Technology* 10(1).
- Douglas, D., Peucker, T. (1973); Visvalingam, M., Whyatt, J. D. (1993);
  Meratnia, N., de By, R. A. (2004). Spatiotemporal compression techniques for moving
  point objects. *EDBT 2004*.
- González, M. C., Hidalgo, C. A., Barabási, A.-L. (2008). Understanding individual human
  mobility patterns. *Nature* 453.

## Citation

If you use trackclean in research, please cite it using [CITATION.cff](CITATION.cff).

## License

MIT © Yusuf Eminoğlu
