stoPTools 0.1.0

Read, process, validate, match public transport stop data from different sources.

stoPTools Python package

The stoPTools Python package provides functions for processing public transport stop data from different sources, including data provided by the German DELFI e.V., PTSA and OpenStreetMap. The package provides routines for

• reading stop data from DELFI ZHV, GTFS feeds, PTSA,
• writing stop data to JSON files,
• validation of DELFI ZHV data,
• validation of GTFS data with focus on typical issues of the DELFI GTFS feed,
• matching stops from different sources.

There're additional scripts (not part of the actual Python package) for

• bulk downloading and processing data from multiple sources,
• viewing processed data on a map,
• matching stops from DELFI ZHV, GTFS, PTSA (API demo only).

The main branch holds the most recent release. The dev branch holds current (usable) state of development.

Install

Run

pip install stoptools

to install the package.

If you want to use the additional scripts, clone the repo and (optionally) do an editable install:

git clone https://codeberg.org/openPTdata/stoPTools.git
pip install -e ./stoPTools

Usage of update_all.py script

The update_all.py script downloads public transport stop data from different sources and converts the data to JSON files. Steps to follow:

1. Rename update_config.json.template to update_config.json. The update_all.py script will read configuration from this file.
2. Adapt values in update_config.json to your needs. Some hints:
   • skip_providers is a list of data provider names to skip. Thus, you may have more providers in your config file than you actually use.
   • providers is a list of data provider definitions:
     • url either is the direct URL to a ZIP archive containing the data or it's the URL of an HTML file containing a link to the ZIP archive. In the latter case set url_regex to a regular expression for extracting the ZIP archive's URL from the HTML code (escape special characters to get a valid JSON string).
     • zip_path is a directory the downloaded ZIP file will be written to.
     • json_path is a directory the processed data will be written to.
     • subsets is a list of subset definitions. For each subset a separate JSON file will be generated. A subset definition is a tuple of subset name, subset type, subset parameters. Valid subset types are all (include all data), bbox (only include data within a bounding box), auth (provider type zhv only; only include data from a specified authority). For bbox subset type the subset parameter is a list of four floats (south, west, north, east boundary of the box in degrees). For auth subset type the subset parameter is a string specifying the authority of interest.
     • For provider type ptsa you have to specify PTSA regions to download data for and you have to provide an Overpass API instance. Don't use public Overpass API instances because the update_all.py script will download all public transport stop related data of the chosen regions in several quite large requests.
3. Run python update_all.py and wait till finished.

Usage of viewer.py script

The viewer.py script generates MBTiles and a Leaflet map from processed data (JSON files). Steps to follow:

1. Use the stoPTools package to generate JSON files with stop data.
2. Rename viewer_config.json.template to viewer_config.json. The viewer.py script will read configuration from this file.
3. Adapt values in viewer_config.json to your needs. Some hints:
   • tmp_path will hold temporary files (GeoJSON and MBTiles) during tile generation. You have to manually delete those files. Depending on your data sources this directory may take several GB of disk space.
   • html_path is where generated HTML files go to. This should be a directory on a (local) web server.
   • tiles_path is where the MBTiles file goes to. This file contains all data (vector tiles) to show on the map and has to be accessible by your tile server (see next item).
   • tiles_url is where Leaflet finds the map tiles to render. You need a tile server to serve the generated MBTiles file's contents under this URL. A good choice is Martin. See Martin Tile Server Documentation for install instructions.
   • skip_data is a list of data item IDs to skip. Thus, you may have more data items in your config file than you acutally use.
   • data is a list of data source configurations including source file, colors aso.
   • wms_layers is a list of aerial imagery sources to add to the map.
4. Run python viewer.py and wait till finished.
5. Start your tile server with the generated MBTiles file.
6. Load the generated HTML file in your web browser (via your web server to avoid problems with CORS).

Usage of match_demo.py script

The match_demo.py script calls various API functions to match stops from DELFI ZHV, (DELFI) GTFS and PTSA. It also contains some code for statistics and post-processing. Matching is done in four steps:

1. prematching (normalize data obtained from different sources and collect possible matches),
2. scoring (compute a score for each prematched stop pair and each attribute),
3. total scores (aggregate attribute scores to one score per stop pair),
4. grouping (group stops by total score if total score is above a minimum score).

Steps to follow:

1. Use the stoPTools package to generate JSON files with stop data.
2. Rename match_demo_config.json.template to match_demo_config.json. The match_demo.py script will read configuration from this file.
3. Adapt values in match_demo_config.json to your needs. Some hints:
   • prematch allows to skip the prematching step (if false) to save computation time if several scoring variants shall be tested.
   • match_data_path and match_scores_path point to CSV files to store prematching and scoring results in. Set prematch to true the first time you run the script to create the CSV files.
4. Run python match_demo.py.

Contributing

Contributions welcome!

To contribute code please open a pull request. To report a bug or to propose an enhancement please file an issue.

See below for contact details of the package maintainer.

License

stoPTools is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the license, or any later version.

Contact and Support

The package is developed and maintained by Jens Flemming.

Feel free to reach out to the developer if you need assistance. Basic documentation may be found in the source code.

For bug reports and enhancement proposals please use the issue tracker.

Funding

Development of this Python package started when the author was head of the VeriBus project. VeriBus was a government-funded project at Zwickau University of Applied Sciences. Funding was provided by the German Federal Ministry of Transport from 2024 till 2026. Development now is based at Hochschule für Technik und Wirtschaft Dresden - University of Applied Sciences.

For legal reasons, we state the following funding information in German:

Das Projekt VeriBus wird im Rahmen der Innovationsinitiative mFUND mit insgesamt 158.464,31 Euro durch das Bundesministerium für Verkehr gefördert.