MTA-Scan — online anomaly detection on live transit feeds
Seven live GTFS-Realtime feeds, an online model that retrains itself with zero labels, and a published table where it loses on false-alarm rate.
River (PARegressor + HalfSpaceTrees), ADWIN drift reset, rolling-quantile calibration with zero labels, a PyTorch shadow model in A/B, TimescaleDB hypertables. A retention-aware checkpoint saver was written after an unbounded save once ate 37 GB.
Results
Every value links to the committed artifact that produces it.
| Metric | Value | Evidence |
|---|---|---|
| Recall@20 on the replay set (216 rows, 16 positives, 3 incidents)a sanity evaluation, not a benchmark — the set size is stated beside the number | 1.00 | docs/generated/replay/metrics.json |
| Average lead time | 6.67 min | docs/generated/replay/metrics.json |
| False-alarm rateworse than the z-score baseline (0.010) — it trades precision for lead time | 0.030 | docs/generated/replay/metrics.json |
What didn’t work
On false-alarm rate the online model loses to the simple baselines. That is in the same published table.
Reported as a null, not hidden — the same honesty policy applies to every number on this site.
Running it
A live map deployment is gated on a TimescaleDB retention policy and a Mapbox token; until both are in place the recorded replay results and the repo stand in for it.