underground 0.3.1

Utilities for NYC's realtime MTA data feeds.

Python MTA Utilities

This is a set of Python utilities that I use to deal with real-time NYC subway data.

I usually want to know when trains are going to depart a specific stop along a specific train line, so right now the tools are mostly for that. But I tried to write them to support arbitrary functionality.

Install

pip install underground

Or if you'd like to live dangerously:

pip install git+https://github.com/nolanbconaway/underground.git#egg=underground

To request data from the MTA, you'll also need a free API key. Register here.

Version 0.2.7.4 vs 0.3.0

On May 1 2020, the MTA is sunsetting the datamine.mta.info service. The new API (api.mta.info) provides identical data but with a new request API.

Users of 0.2.x will need to migrate by doing the following:

1. Get a new API key at api.mta.info. This key is longer than the one provided by the datamine API. Underground understands this key in the same way as the old one.
2. Replace all feed IDs with route IDs or URLs. The feed IDs have changed for the new API, and not all feeds have IDs any more. Version 0.3 of Underground was built with a route_or_url concept for feed selection. Users may provide the URL for the feed they want (see this page), or they may provide a route ID (in which case the appropriate URL is then selected).

Code from v0.2.x such as this:

feed = SubwayFeed.get(metadata.get_feed_id('Q'))

Now becomes in v0.3:

# under the hood, the correct URL is selected.
feed = SubwayFeed.get('Q')

# or 
url = 'https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-nqrw'
feed = SubwayFeed.get(url)

Python API

Once you have your API key, use the Python API like:

import os

from underground import metadata, SubwayFeed

API_KEY = os.getenv('MTA_API_KEY')
ROUTE = 'Q'
feed = SubwayFeed.get(ROUTE, api_key=API_KEY)

# request will read from $MTA_API_KEY if a key is not provided
feed = SubwayFeed.get(ROUTE)

# under the hood, the Q route is mapped to a URL. This call is equivalent:
URL = 'https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-nqrw'
feed = SubwayFeed.get(URL)

# or
URL = metadata.resolve_url(ROUTE)
feed = SubwayFeed.get(URL)

List train stops on each line

feed.extract_stop_dict will return a dictionary of dictionaries, like:

>>> feed.extract_stop_dict()

{

  "route_1": {
    "stop_1": [datetime.datetime(...), datetime.datetime(...)], 
    "stop_2": [datetime.datetime(...), datetime.datetime(...)], 
    ...
  }, 
  "route_2": {
    "stop_1": [datetime.datetime(...), datetime.datetime(...)], 
    "stop_2": [datetime.datetime(...), datetime.datetime(...)], 
    ...
  }

}

CLI

The underground command line tool is also installed with the package.

$ underground --help
Usage: underground [OPTIONS] COMMAND [ARGS]...

  Command line handlers for MTA realtime data.

Options:
  --help  Show this message and exit.

Commands:
  feed       Request an MTA feed.
  findstops  Find your stop ID.
  stops      Print out train departure times for all stops on a subway line.
  version    Print the underground version.

feed

$ underground feed --help
Usage: underground feed [OPTIONS] ROUTE_OR_URL

  Request an MTA feed via a route or URL.

  ROUTE_OR_URL may be either a feed URL or a route (which will be used to
  look up the feed url).

  Examples (both access the same feed):

      underground feed Q --json > feed_nrqw.json

      URL='https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-nqrw' &&
      underground feed $URL --json > feed_nrqw.json

Options:
  --api-key TEXT         MTA API key. Will be read from $MTA_API_KEY if not
                         provided.

  --json                 Option to output the feed data as JSON. Otherwise
                         output will be bytes.

  -r, --retries INTEGER  Retry attempts in case of API connection failure.
                         Default 100.

  --help                 Show this message and exit.

stops

$ underground stops --help
Usage: underground stops [OPTIONS] [H|M|D|1|Z|A|N|GS|SI|J|G|Q|L|B|R|F|E|2|7|W|
                          6|4|C|5|FS]

  Print out train departure times for all stops on a subway line.

Options:

  -f, --format TEXT      strftime format for stop times. Use `epoch` for a
                          unix timestamp.
  -r, --retries INTEGER  Retry attempts in case of API connection failure.
                          Default 100.
  --api-key TEXT         MTA API key. Will be read from $MTA_API_KEY if not
                          provided.
  -t, --timezone TEXT    Output timezone. Ignored if --epoch. Default to NYC
                          time.
  --help                 Show this message and exit.

Stops are printed to stdout in the format stop_id t1 t2 ... tn .

$ export MTA_API_KEY='...'
$ underground stops Q | tail -2
Q05S 19:01 19:09 19:16 19:25 19:34 19:44 19:51 19:58
Q04S 19:03 19:11 19:18 19:27 19:36 19:46 19:53 20:00

If you know your stop id (stop IDs can be found in stops.txt), you can grep the results:

$ underground stops Q | grep Q05S
Q05S 19:09 19:16 19:25 19:34 19:44 19:51 19:58

If you don't know your stop, see below for a handy tool!

findstops

$ underground findstops --help
Usage: underground findstops [OPTIONS] QUERY...

  Find your stop ID.

  Query a location and look for your stop ID, like:

  $ underground findstops parkside av

Options:

  --json  Option to output the data as JSON. Otherwise will be human readable
          table.

  --help  Show this message and exit.

Enter the name of your stop and a table of stops with matching names will be returned.

$ underground findstops parkside
ID: D27N    Direction: NORTH    Lat/Lon: 40.655292, -73.961495    Name: PARKSIDE AV
ID: D27S    Direction: SOUTH    Lat/Lon: 40.655292, -73.961495    Name: PARKSIDE AV

Some names are ambiguous (try "fulton st"), for these you'll have to dig into the metadata more carefully.