Metadata-Version: 2.5
Name: microsegments
Version: 0.2.18
Summary: Where do transit vehicles linger? Count vehicle position observations per micro-segment of a line, from GTFS + AVL / GTFS-RT positions.
Project-URL: Homepage, https://github.com/GaspardMerten/microsegments
Project-URL: Repository, https://github.com/GaspardMerten/microsegments
Project-URL: Issues, https://github.com/GaspardMerten/microsegments/issues
Author-email: Gaspard Merten <gaspard.mp.work@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: avl,bus,gtfs,gtfs-rt,hotspots,polars,tram,transit
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: gtfs-parquet>=0.7
Requires-Dist: numpy>=1.26
Requires-Dist: polars>=1.0
Requires-Dist: shapely>=2.0
Provides-Extra: dev
Requires-Dist: hypothesis>=6.0; extra == 'dev'
Requires-Dist: matplotlib>=3.8; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Provides-Extra: gtfsrt
Requires-Dist: gtfs-realtime-bindings>=1.0; extra == 'gtfsrt'
Provides-Extra: mobilitytwin
Requires-Dist: requests>=2.31; extra == 'mobilitytwin'
Provides-Extra: plot
Requires-Dist: matplotlib>=3.8; extra == 'plot'
Description-Content-Type: text/markdown

# microsegments

Where do buses and trams linger? `microsegments` cuts each line into short stretches (30 m by default)
and counts how many times a vehicle position is observed in each one, per hour and per day of the week.
It works with GTFS plus any vehicle position feed: GTFS-RT VehiclePositions (lat/lon, as CSV, parquet or
`.pb`) or linear-referenced AVL (last stop + distance, like STIB).

![Report page: hour player, map coloured by micro-segment, hotspots, hour x micro-segment matrix](docs/report.jpg)

## Install

```bash
pip install "microsegments[plot]"        # plot extra = matplotlib
```

Python ≥ 3.11. Core dependencies: polars, numpy, shapely, gtfs-parquet (no pandas).

## Quick start (offline, synthetic data)

```bash
python examples/make_synth.py                     # simulated tram line S1 -> examples/data/
microsegments run examples/synth.toml -o out/     # result.parquet, analysis.json, report.html, matrix_dir*.png
```

## Linear-referenced feeds (STIB vehicle-distance)

```toml
[input]
paths = ["vd55/*.parquet"]           # <YYYYMMDD>.parquet (ts, dir, point, dist) + <YYYYMMDD>.snaps.parquet
kind = "stib"                        # or "linear" for any "stop_id + dist_m" table (map columns below)
events = "punctuality/*.parquet"     # optional stop events: passages from both sources, max taken

[input.linear]
id_normaliser = "leading_digits"     # "05766" -> "5766"
feed_length = "auto"                 # rescale feed metres to GTFS link lengths

[gtfs]
path = "gtfs/"                       # zip, .txt directory or gtfs-parquet directory
# dated = "gtfs/{date}"              # one feed per day instead

[select]
route = "55"
dates = "2025-02-17..2025-05-16"
weekdays = [0, 1, 2, 3, 4]
```

See `examples/stib55.toml` (runs on the two-day test fixture).

## GTFS-RT positions as CSV (lat/lon)

```toml
[input]
paths = ["positions/*.csv"]
kind = "latlon"
ts_unit = "s"                        # "s" | "ms" | "us" | "iso"
[input.columns]                      # canonical name = your column
ts = "timestamp"
route_id = "route_id"
vehicle_id = "vehicle_id"
lat = "latitude"
lon = "longitude"
trip_id = "trip_id"                  # optional, helps map matching
```

Per-vehicle fixes are thinned to one per `tick_s` (20 s) so counts stay comparable with a snapshot feed.
See `examples/gtfsrt_csv.toml` for every option.

## Python

```python
import microsegments as ms

res = ms.run(ms.Config.from_toml("ms.toml"))   # obs -> coverage -> network -> place -> passages
                                                # -> segments -> count -> analyse -> hotspots
res.result            # polars: one row per segment x hour (bands as hour -1 am, -2 pm, -3 day, -4 evening)
res.hotspots          # ranked stretches
ms.export(res, "report.html", lang="fr")       # standalone page (fr / en)
ms.plot.matrix(res, direction_id=0, metric="obs_per_passage", days=[0, 1, 2, 3, 4], stops=False)
ms.plot.map(res, 0, "pm", metric="excess_per_passage")
contract = res.contract()                      # the page JSON (also the platform's /api/analysis)

pre = ms.prepare(cfg)                          # reuse placement to tune the segment length
tr = ms.tune.segment_length(pre.placed, pre.segment_fn(), coverage=pre.coverage, passages=pre.passages,
                            pattern_days=pre.network.pattern_days)
ms.plot.tune_curves(tr)
```

Stages are usable on their own: `read_observations`, `build_network`, `segment`, `count`, `analyse`,
`hotspots`, `simulate`.

## CLI

```bash
microsegments run CONFIG -o OUT [--lang en] [--segment-m 20] [--dates A..B] [--source "credit"]
microsegments tune CONFIG [--lengths 10,20,30,50] [--sensitivity] [-o OUT]
microsegments hotspots CONFIG [-o hotspots.geojson|.csv|.parquet]
microsegments inspect CONFIG        # coverage per day, GTFS versions, placement, evening vs line reference
microsegments compare CONFIG --a 2025-02-17..2025-03-31 --b 2025-04-01..2025-05-15 -o OUT
microsegments sensitivity-table CONFIG [CONFIG ...] -o OUT    # one tidy table + markdown over several lines
```

## What the numbers mean

Every poll (about every 20 s) puts each vehicle in one micro-segment: one **observation**. One observation
is therefore about 20 s of presence.

| Metric | Definition | Read as |
|---|---|---|
| `obs_per_passage` (default) | Σ observations / Σ passages | × 20 s ≈ time each vehicle spends there; independent of the frequency |
| `obs_per_h` | Σ observations / Σ covered hours | vehicles seen in the segment per hour of polling; depends on slowness and frequency (`obs_per_h_10m`: per 10 m, Python API only) |
| `excess_per_passage` | obs per passage − the same at 20–23 h, same days | extra observations (≈ seconds / 20) lost per vehicle vs free-flowing evening |
| `excess_obs_per_h`, `log2_ratio` | see `metrics.py` | |

All values are ratios of sums over the selected days (optionally post-stratified by weekday). Stop zones
(30 m before to 60 m after each stop; with `grid="stops"`, bins of their own whose extent can be set per
link, e.g. from where a STIB vehicle switches to the stop to its exit mark) can be masked to bring out
signals and junctions. The colour scale is
one fixed function of the value for every line, period, hour and option (`microsegments.scale`): values are
brought to seconds per passage per 30 m of track and read on the STIB speed ramp (green 4 s ≈ 27 km/h, red
14 s ≈ 7.5 km/h, dark red ≥ 24 s) on a linear legend axis from 3 to 24 s; both excesses share it as 3 s + excess. The page
always shows values per 30 m and defaults to time spent per vehicle.

## Missing data and GTFS versions

- **Hours** with < 80 % of polls, a frozen feed (> 5 min), the line absent, or abnormal vehicle counts are
  dropped from numerator and denominator. Polling gaps are handled by the exposure (covered hours); counts
  are never imputed.
- **Days** with more than 25 % of their 6–21 h dropped are excluded and listed with the reason
  (`res.analysis.excluded_days`, coverage calendar in the page).
- **GTFS changes**: patterns are rebuilt per service date; `link_key` / `seg_key` are stable across
  versions. Each segment is aggregated only over the days it belongs to the day's main pattern (the
  reference too). The page shows the most frequent version; others are selectable. A single feed whose
  calendar does not cover some dates lends them the patterns of the same weekday.

## Choosing the segment length

`microsegments tune` scores L ∈ {10, 15, 20, 30, 40, 50, 75, 100} m and picks the smallest L whose
**time lost per vehicle** (day profile) is reproducible: split-half reliability ≥ 0.8 (days split at
random into two halves stratified by weekday, Spearman-Brown corrected) and hotspot localisation spread
≤ 30 m. When no length passes, the default 30 m is kept. The leave-one-day-out Poisson deviance (and its
1-SE rule), the obs/h reliability and the hotspot Jaccard are reported as diagnostics only. 30 m is a good
default for 20 s polling at urban speeds; go shorter only with dense GTFS-RT fixes and many days. `--sensitivity`
re-runs phase offsets, gap caps (30, 40, 60, ∞ s), stop zones, references, passage sources and segment
lengths 15 / 30 / 60 m around the baseline.

## Hotspots

Stretches where the excess over the evening has a 95 % bootstrap lower bound above 2 s per vehicle, is
positive on ≥ 60 % of days, survives Benjamini–Hochberg (q = 0.1) and lasts ≥ 2 hours or a peak band.
Adjacent bins merge, never across a stop / running border. Classes: *infrastructure* (all day),
*congestion* (peak only), *mixed*. Needs at least 5 included days. Export as GeoJSON with
`microsegments hotspots CONFIG -o hotspots.geojson`.

## Comparing two periods

`microsegments compare` (or `ms.compare(analysis_a, analysis_b)`) compares observations per passage
segment by segment and hour by hour, each period with its own days and references, with a day bootstrap
of the difference. Segments are matched by key (or by stop pair when a new GTFS version moved the shape a
little); the others are listed and left out. Adjacent significant segments form stretches, ranked by the
change in vehicle time per day; stretches whose frequency changed by more than 20 % are flagged, since a
timetable change can explain part of the difference. The HTML page gets a "compare two periods" view.

## License

MIT. The bundled Leaflet CSS is BSD-2-Clause.
