gtfs-parquet

Parse GTFS feeds to/from Parquet via Polars — fast, compact, typed.

Features

• Parse GTFS from zip, directory, or URL
• Read NeTEx timetables (European profile) as the same GTFS tables
• Write to Parquet (zstd-compressed) or back to GTFS
• Strongly typed schemas with optimised dtypes (Float32 coords, Int16 sequences)
• Built on Polars — zero-copy reads, lazy evaluation ready
• Operations: calendar expansion, network analysis, route/stop/trip stats, timetable graphs, CSA connections

Installation

pip install gtfs-parquet

Quick start

from gtfs_parquet import parse_gtfs, write_parquet, read_parquet, write_gtfs

# Parse a GTFS zip (local path or URL)
feed = parse_gtfs("gtfs.zip")

# Write to Parquet — directory, .zip, or .tar (auto-detected by extension)
write_parquet(feed, "output/")           # directory of .parquet files
write_parquet(feed, "output.zip")        # zip archive (no extra compression)
write_parquet(feed, "output.tar")        # tar archive (single file, no extra compression)

# Read back (same formats)
feed = read_parquet("output/")
feed = read_parquet("output.zip")
feed = read_parquet("output.tar")

# Convert back to GTFS zip
write_gtfs(feed, "roundtrip.zip")

Additional top-level helpers: parse_gtfs_dir, parse_gtfs_zip, write_gtfs_dir, and to_parquet_bytes(feed), which returns {table_name: parquet_bytes} without writing to disk (handy for uploading to object storage).

Large feeds

parse_gtfs_zip loads the whole feed in memory. For a national feed that takes several GB of RAM. When you only need the Parquet files, convert_gtfs_zip streams the conversion instead:

from gtfs_parquet import convert_gtfs_zip

paths = convert_gtfs_zip("gtfs.zip", "output/")   # {"stops": Path("output/stops.parquet"), ...}

Tables are converted one at a time, and large files are read in 16 MB blocks, stored as temporary Parquet files in out_dir until the table is written. Rows keep the order of the source file (sort=True sorts them). A file with an unescaped quote (12" Street outside quotes) raises a ValueError instead of being read whole.

Memory grows with the number of Polars threads, which defaults to the number of CPU cores: set POLARS_MAX_THREADS before importing Polars to limit it.

NeTEx

Some operators publish their timetable only in NeTEx, the European XML format. parse_netex reads it as GTFS tables, typed like a parsed GTFS feed; convert_netex writes them as Parquet, the same files convert_gtfs_zip writes:

from gtfs_parquet import convert_netex, parse_netex

feed = parse_netex("IT-IT-TRENITALIA_L1.xml.gz")   # .xml, .xml.gz, or a .zip of XML files
paths = convert_netex("IT-IT-TRENITALIA_L1.xml.gz", "output/")

The file is read in one streaming pass. Trenitalia's timetable (302 MB of XML, 16,675 journeys) converts in about 10 seconds with 0.6 GB of RAM.

Compression

Parquet output is significantly smaller than the original GTFS zip thanks to zstd compression and optimised column types:

Since 0.6.0, rows keep the order of the source files by default: feeds usually list related rows together, and sorting by id often breaks that up.

Feed object

Feed is a plain dataclass with one optional polars.DataFrame attribute per GTFS file (e.g. feed.stops, feed.routes, feed.stop_times). Only files present in the source feed are populated.

feed.tables()    # dict of all non-None tables
feed.validate()  # check required files and columns

Operations

All operations are standalone functions that take a Feed as their first argument. Import from the ops submodules:

from gtfs_parquet.ops.calendar import (
    get_dates, get_first_week, get_week, get_active_services,
    subset_dates, compute_trip_activity, compute_busiest_date,
)
from gtfs_parquet.ops.trips import get_trips, compute_trip_stats
from gtfs_parquet.ops.routes import get_routes, build_route_timetable, compute_route_stats
from gtfs_parquet.ops.stops import (
    get_stops, get_stop_times, get_start_and_end_times,
    build_stop_timetable, compute_stop_activity, compute_stop_stats,
)
from gtfs_parquet.ops.network import describe, compute_network_stats
from gtfs_parquet.ops.restrict import restrict_to_routes, restrict_to_dates, restrict_to_trips
from gtfs_parquet.ops.clean import clean, clean_ids, drop_zombies
from gtfs_parquet.ops.graph import (
    build_timetable_graph, get_service_day_counts, build_stop_lookup,
    compute_segment_frequencies, compute_connections, served_stations,
)

dates = get_dates(feed)
week = get_first_week(feed)
services = get_active_services(feed, dates[0])

trip_stats = compute_trip_stats(feed)
route_stats = compute_route_stats(feed, [dates[0]], trip_stats)
stop_stats = compute_stop_stats(feed, [dates[0]])

# Timetable graph for routing
graph = build_timetable_graph(feed, services, hour_filter=(6, 22))

# CSA-compatible connections
connections = compute_connections(feed, services)

# Segment frequencies weighted by service days
day_counts = get_service_day_counts(feed, dates)
freqs = compute_segment_frequencies(feed, services, service_day_counts=day_counts)

The API is inspired by gtfs-kit, re-implemented on Polars for significantly better performance.

Performance vs gtfs-kit

Benchmarked on the STIB (Brussels) feed (~5.5 MB, ~9 000 trips):

Peak process memory: 1020 MB (gtfs-kit) vs 744 MB (gtfs-parquet).

License

MIT