gtfs-parquet

Parse GTFS feeds to/from Parquet via Polars — fast, compact, typed.

Features

• Parse GTFS from zip, directory, or URL
• Write to Parquet (zstd-compressed) or back to GTFS
• Strongly typed schemas with optimised dtypes (Float32 coords, Int16 sequences)
• Built on Polars — zero-copy reads, lazy evaluation ready
• Operations: calendar expansion, network analysis, route/stop/trip stats, timetable graphs, CSA connections

Installation

pip install gtfs-parquet

Quick start

from gtfs_parquet import parse_gtfs, write_parquet, read_parquet, write_gtfs

# Parse a GTFS zip (local path or URL)
feed = parse_gtfs("gtfs.zip")

# Write to Parquet — directory, .zip, or .tar (auto-detected by extension)
write_parquet(feed, "output/")           # directory of .parquet files
write_parquet(feed, "output.zip")        # zip archive (no extra compression)
write_parquet(feed, "output.tar")        # tar archive (single file, no extra compression)

# Read back (same formats)
feed = read_parquet("output/")
feed = read_parquet("output.zip")
feed = read_parquet("output.tar")

# Convert back to GTFS zip
write_gtfs(feed, "roundtrip.zip")

Additional top-level helpers: parse_gtfs_dir, parse_gtfs_zip, write_gtfs_dir, and to_parquet_bytes(feed), which returns {table_name: parquet_bytes} without writing to disk (handy for uploading to object storage).

Large feeds

parse_gtfs_zip loads the whole feed in memory. For a national feed that takes several GB of RAM. When you only need the Parquet files, convert_gtfs_zip streams the conversion instead:

from gtfs_parquet import convert_gtfs_zip

paths = convert_gtfs_zip("gtfs.zip", "output/")   # {"stops": Path("output/stops.parquet"), ...}

Tables are converted one at a time, and large files are read in 16 MB blocks, stored as temporary Parquet files in out_dir until the table is written. Rows keep the order of the source file (sort=True sorts them). A file with an unescaped quote (12" Street outside quotes) raises a ValueError instead of being read whole.

Memory grows with the number of Polars threads, which defaults to the number of CPU cores: set POLARS_MAX_THREADS before importing Polars to limit it.

Compression

Parquet output is significantly smaller than the original GTFS zip thanks to zstd compression and optimised column types:

Since 0.6.0, rows keep the order of the source files by default: feeds usually list related rows together, and sorting by id often breaks that up.

Feed object

Feed is a plain dataclass with one optional polars.DataFrame attribute per GTFS file (e.g. feed.stops, feed.routes, feed.stop_times). Only files present in the source feed are populated.

feed.tables()    # dict of all non-None tables
feed.validate()  # check required files and columns

Operations

All operations are standalone functions that take a Feed as their first argument. Import from the ops submodules:

from gtfs_parquet.ops.calendar import (
    get_dates, get_first_week, get_week, get_active_services,
    subset_dates, compute_trip_activity, compute_busiest_date,
)
from gtfs_parquet.ops.trips import get_trips, compute_trip_stats
from gtfs_parquet.ops.routes import get_routes, build_route_timetable, compute_route_stats
from gtfs_parquet.ops.stops import (
    get_stops, get_stop_times, get_start_and_end_times,
    build_stop_timetable, compute_stop_activity, compute_stop_stats,
)
from gtfs_parquet.ops.network import describe, compute_network_stats
from gtfs_parquet.ops.restrict import restrict_to_routes, restrict_to_dates, restrict_to_trips
from gtfs_parquet.ops.clean import clean, clean_ids, drop_zombies
from gtfs_parquet.ops.graph import (
    build_timetable_graph, get_service_day_counts, build_stop_lookup,
    compute_segment_frequencies, compute_connections, served_stations,
)

dates = get_dates(feed)
week = get_first_week(feed)
services = get_active_services(feed, dates[0])

trip_stats = compute_trip_stats(feed)
route_stats = compute_route_stats(feed, [dates[0]], trip_stats)
stop_stats = compute_stop_stats(feed, [dates[0]])

# Timetable graph for routing
graph = build_timetable_graph(feed, services, hour_filter=(6, 22))

# CSA-compatible connections
connections = compute_connections(feed, services)

# Segment frequencies weighted by service days
day_counts = get_service_day_counts(feed, dates)
freqs = compute_segment_frequencies(feed, services, service_day_counts=day_counts)

The API is inspired by gtfs-kit, re-implemented on Polars for significantly better performance.

Performance vs gtfs-kit

Benchmarked on the STIB (Brussels) feed (~5.5 MB, ~9 000 trips):

Peak process memory: 1020 MB (gtfs-kit) vs 744 MB (gtfs-parquet).

License

MIT