microsegments

Where do buses and trams linger? microsegments cuts each line into short stretches (30 m by default) and counts how many times a vehicle position is observed in each one, per hour and per day of the week. It works with GTFS plus any vehicle position feed: GTFS-RT VehiclePositions (lat/lon, as CSV, parquet or .pb) or linear-referenced AVL (last stop + distance, like STIB).

Install

pip install "microsegments[plot]"        # plot extra = matplotlib

Python ≥ 3.11. Core dependencies: polars, numpy, shapely, gtfs-parquet (no pandas).

Quick start (offline, synthetic data)

python examples/make_synth.py                     # simulated tram line S1 -> examples/data/
microsegments run examples/synth.toml -o out/     # result.parquet, analysis.json, report.html, matrix_dir*.png

Linear-referenced feeds (STIB vehicle-distance)

[input]
paths = ["vd55/*.parquet"]           # <YYYYMMDD>.parquet (ts, dir, point, dist) + <YYYYMMDD>.snaps.parquet
kind = "stib"                        # or "linear" for any "stop_id + dist_m" table (map columns below)
events = "punctuality/*.parquet"     # optional stop events: passages from both sources, max taken

[input.linear]
id_normaliser = "leading_digits"     # "05766" -> "5766"
feed_length = "auto"                 # rescale feed metres to GTFS link lengths

[gtfs]
path = "gtfs/"                       # zip, .txt directory or gtfs-parquet directory
# dated = "gtfs/{date}"              # one feed per day instead

[select]
route = "55"
dates = "2025-02-17..2025-05-16"
weekdays = [0, 1, 2, 3, 4]

See examples/stib55.toml (runs on the two-day test fixture).

GTFS-RT positions as CSV (lat/lon)

[input]
paths = ["positions/*.csv"]
kind = "latlon"
ts_unit = "s"                        # "s" | "ms" | "us" | "iso"
[input.columns]                      # canonical name = your column
ts = "timestamp"
route_id = "route_id"
vehicle_id = "vehicle_id"
lat = "latitude"
lon = "longitude"
trip_id = "trip_id"                  # optional, helps map matching

Per-vehicle fixes are thinned to one per tick_s (20 s) so counts stay comparable with a snapshot feed. See examples/gtfsrt_csv.toml for every option.

Python

import microsegments as ms

res = ms.run(ms.Config.from_toml("ms.toml"))   # obs -> coverage -> network -> place -> passages
                                                # -> segments -> count -> analyse -> hotspots
res.result            # polars: one row per segment x hour (bands as hour -1 am, -2 pm, -3 day, -4 evening)
res.hotspots          # ranked stretches
ms.export(res, "report.html", lang="fr")       # standalone page (fr / en)
ms.plot.matrix(res, direction_id=0, metric="obs_per_h", days=[0, 1, 2, 3, 4], stops=False)
ms.plot.map(res, 0, "pm", metric="excess_per_passage")
contract = res.contract()                      # the page JSON (also the platform's /api/analysis)

pre = ms.prepare(cfg)                          # reuse placement to tune the segment length
tr = ms.tune.segment_length(pre.placed, pre.segment_fn(), coverage=pre.coverage, passages=pre.passages,
                            pattern_days=pre.network.pattern_days)
ms.plot.tune_curves(tr)

Stages are usable on their own: read_observations, build_network, segment, count, analyse, hotspots, simulate.

CLI

microsegments run CONFIG -o OUT [--lang en] [--segment-m 20] [--dates A..B] [--source "credit"]
microsegments tune CONFIG [--lengths 10,20,30,50] [--sensitivity] [-o OUT]
microsegments hotspots CONFIG [-o hotspots.geojson|.csv|.parquet]
microsegments inspect CONFIG        # coverage per day, GTFS versions, placement, evening vs line reference
microsegments compare CONFIG --a 2025-02-17..2025-03-31 --b 2025-04-01..2025-05-15 -o OUT
microsegments sensitivity-table CONFIG [CONFIG ...] -o OUT    # one tidy table + markdown over several lines

What the numbers mean

Every poll (about every 20 s) puts each vehicle in one micro-segment: one observation. One observation is therefore about 20 s of presence.

All values are ratios of sums over the selected days (optionally post-stratified by weekday). Stop zones (30 m before to 60 m after each stop) can be masked to bring out signals and junctions. The colour scale is one fixed function of the value for every line, period, hour and option (microsegments.scale): values are brought to seconds per passage per 30 m of track and read on the STIB speed ramp (green 4 s ≈ 27 km/h, red 14 s ≈ 7.5 km/h, dark red ≥ 24 s); both excesses share it as 3 s + excess.

Missing data and GTFS versions

• Hours with < 80 % of polls, a frozen feed (> 5 min), the line absent, or abnormal vehicle counts are dropped from numerator and denominator. Polling gaps are handled by the exposure (covered hours); counts are never imputed.
• Days with more than 25 % of their 6–21 h dropped are excluded and listed with the reason (res.analysis.excluded_days, coverage calendar in the page).
• GTFS changes: patterns are rebuilt per service date; link_key / seg_key are stable across versions. Each segment is aggregated only over the days it belongs to the day's main pattern (the reference too). The page shows the most frequent version; others are selectable. A single feed whose calendar does not cover some dates lends them the patterns of the same weekday.

Choosing the segment length

microsegments tune scores L ∈ {10, 15, 20, 30, 40, 50, 75, 100} m (leave-one-day-out Poisson deviance, split-half reliability, hotspot localisation spread and Jaccard) and picks the smallest L with reliability ≥ 0.8, spread ≤ 30 m and deviance within one standard error of the minimum. 30 m is a good default for 20 s polling at urban speeds; go shorter only with dense GTFS-RT fixes and many days. --sensitivity re-runs phase offsets, gap caps (30, 40, 60, ∞ s), stop zones, references, passage sources and segment lengths 15 / 30 / 60 m around the baseline.

Hotspots

Stretches where the excess over the evening has a 95 % bootstrap lower bound above 2 s per vehicle, is positive on ≥ 60 % of days, survives Benjamini–Hochberg (q = 0.1) and lasts ≥ 2 hours or a peak band. Adjacent bins merge, never across a stop / running border. Classes: infrastructure (all day), congestion (peak only), mixed. Needs at least 5 included days. Export as GeoJSON with microsegments hotspots CONFIG -o hotspots.geojson.

Comparing two periods

microsegments compare (or ms.compare(analysis_a, analysis_b)) compares observations per passage segment by segment and hour by hour, each period with its own days and references, with a day bootstrap of the difference. Segments are matched by key (or by stop pair when a new GTFS version moved the shape a little); the others are listed and left out. Adjacent significant segments form stretches, ranked by the change in vehicle time per day; stretches whose frequency changed by more than 20 % are flagged, since a timetable change can explain part of the difference. The HTML page gets a "compare two periods" view.

License

MIT. The bundled Leaflet CSS is BSD-2-Clause.