path4gmns 0.7.4

An open-source, cross-platform, lightweight, and fast Python path engine for networks encoded in GMNS

Path4GMNS

Path4GMNS is an open-source, cross-platform, lightweight, and fast Python path engine for networks encoded in GMNS. Besides finding the static and time-dependent shortest path for simple analyses, its main functionality is to provide an efficient and flexible framework for column(path)-based modeling and applications in transportation (e.g., activity-based demand modeling). Path4GMNS supports, in short,

1. finding (static) shortest path between two nodes;
2. constructing shortest paths for all individual agents;
3. performing path-based User-Equilibrium (UE) traffic assignment;
4. evaluating multimodal accessibility.

Path4GMNS also serves as an API to the C++-based DTALite to conduct various multimodal traffic assignments including,

• Link-based UE,
• Path-based UE,
• UE + Dynamic Traffic Assignment (DTA),
• Origin-Destination Matrix Estimation (ODME).

Installation

Path4GMNS has been published on PyPI, and can be installed using

$ pip install path4gmns

If you need a specific version of Path4GMNS, say, 0.7.3,

$ pip install path4gmns==0.7.3

v0.7.3 comes with bug fixes and new functionalities. All previous releases shall be deprecated for any purposes.

Dependency

The Python modules are written in Python 3.x, which is the minimum requirement to explore the most of Path4GMNS. Some of its functions require further run-time support, which we will go through along with the corresponding use cases in the following section.

Getting Started

Download the Test Data Set

A sample data set with six different networks are provided. You can manually retrieve each individual test network from here or use the built-in helper function to automatically download the whole data set.

import path4gmns as pg

pg.download_sample_data_sets()

Note that requests (2.21.1 or higher) is needed for you to proceed downloading.

Get the Shortest Path between Two Nodes

Find the (static) shortest path (based on distance) and output it in the format of a sequence of node/link IDs.

import path4gmns as pg

network = pg.read_network(load_demand=False)

print('\nshortest path (node id) from node 1 to node 2, '
      +network.find_shortest_path(1, 2))
print('\nshortest path (link id) from node 1 to node 2, '
      +network.find_shortest_path(1, 2, 'link'))

You can specify the absolute path or the relative path from your cwd in read_network() to use a particular network from the downloaded sample data set.

import path4gmns as pg

network = pg.read_network(load_demand=False, input_dir='data/Chicago_Sketch')

print('\nshortest path (node id) from node 1 to node 2, '
      +network.find_shortest_path(1, 2))
print('\nshortest path (link id) from node 1 to node 2, '
      +network.find_shortest_path(1, 2, 'link'))

Retrieving the shortest path between any two (different) nodes under a specific mode is now available under v0.7.2 or higher.

import path4gmns as pg

network = pg.read_network(load_demand=False)

print('\nshortest path (node id) from node 1 to node 2, '
      +network.find_shortest_path(1, 2, mode='w'))
print('\nshortest path (link id) from node 1 to node 2, '
      +network.find_shortest_path(1, 2, mode='w', seq_type='link'))

The mode passed to find_shortest_path() must be defined in settings.yaml, which could be either the type or the name. Take the above sample code for example, the type 'w' and its name 'walk' are both valid and equivalent to each other. See Perform Multimodal Accessibility Evaluation for more information.

Find Shortest Paths for All Individual Agents

Path4GMNS is capable of calculating and constructing the (static) shortest paths for all agents. Individual agents will be automatically set up using the aggregated travel demand between each OD pair within find_path_for_agents() on its first call.

The unique agent paths can be outputed to a csv file as shown in the example below.

import path4gmns as pg

network = pg.read_network()
network.find_path_for_agents()

agent_id = 300
print('\norigin node id of agent is '
      f'{network.get_agent_orig_node_id(agent_id)}')
print('destination node id of agent is '
      f'{network.get_agent_dest_node_id(agent_id)}')
print('shortest path (node id) of agent, '
      f'{network.get_agent_node_path(agent_id)}')
print('shortest path (link id) of agent, '
      f'{network.get_agent_link_path(agent_id)}')

agent_id = 1000
print('\norigin node id of agent is '
      f'{network.get_agent_orig_node_id(agent_id)}')
print('destination node id of agent is '
      f'{network.get_agent_dest_node_id(agent_id)}')
print('shortest path (node id) of agent, '
      f'{network.get_agent_node_path(agent_id)}')
print('shortest path (link id) of agent, '
      f'{network.get_agent_link_path(agent_id)}')

# output unique agent paths to a csv file
# if you do not want to include geometry info in the output file,
# use pg.output_agent_paths(network, False)
pg.output_agent_paths(network)

v0.7.2 or higher features finding agent paths under a specific mode defined in settings.yaml. The following example demostrates this new functionality under mode walk (i.e., w).

import path4gmns as pg

network = pg.read_network()
network.find_path_for_agents()

# or equivalently network.find_path_for_agents('walk')
network.find_path_for_agents('w')

# retrieving the origin, the destination, and the shortest path of a given agent
# is exactly the same as before as well as outputing all unique agent paths

Perform Path-Based UE Traffic Assignment using the Python Column-Generation Module

The Python column-generation module only implements path-based UE. If you need other assignment modes, e.g., link-based UE or DTA, please use perform_network_assignment_DTALite().

import path4gmns as pg

network = pg.read_network()

assignment_num = 20
column_update_num = 10

# path-based UE only
pg.perform_column_generation(assignment_num, column_update_num, network)

# if you do not want to include geometry info in the output file,
# use pg.output_columns(network, False)
pg.output_columns(network)
pg.output_link_performance(network)

NOTE THAT you can still use the legacy pg.perform_network_assignment(assignment_mode=1, assignment_num, column_update_num, network) to perform the same functionality here. But it has been deprecated, and will be removed later.

Starting from v0.7.0a1, Path4GMNS supports loading columns/paths from existing files (generated from either the Python module or DTALite) and continue the column-generation procedure from where you left. Please skip the assignment stage and go directly to column pool optimization by setting assignment_num = 0.

import path4gmns as pg

network = pg.read_network()
# you can specify the input directory
# e.g., pg.load_columns(network, 'data/Chicago_Sketch')
pg.load_columns(network)

# we recommend NOT doing assignemnt again after loading columns
assignment_num = 0
column_update_num = 10

pg.perform_column_generation(assignment_num, column_update_num, network)

pg.output_columns(network)
pg.output_link_performance(network)

Perform Traffic Assignment using DTALite

DTALite has the following four assignment modes to choose.

  0: Link-based UE
  1: Path-based UE
  2: UE + DTA
  3: ODME

The next example demonstrates how to perform path-based UE (i.e., mode 1) using DTALite from Path4GMNS.

import path4gmns as pg

# no need to call read_network() like the python module
# as network and demand loading will be handled within DTALite

# path-based UE
mode = 1
assignment_num = 10
column_update_num = 10

pg.perform_network_assignment_DTALite(mode, assignment_num, column_update_num)

# no need to call output_columns() and output_link_performance()
# since outputs will be processed within DTALite

print('\npath finding results can be found in agent.csv')

The OpenMP run-time library must be installed to utilize the built-in parallel computing feature in DTALite (and DTALite would not be able to run if the run-time support is absent). Its installation varies by operating systems.

Windows Users

Download Microsoft Visual C++ Redistributable for Visual Studio 2019 and check here for more information and earlier versions.

Linux Users

No actions are needed as most Linux distributions have GCC installed with OpenMP support.

macOS Users

You will need to install libomp using Homebrew.

$ brew install libomp

Perform Multimodal Accessibility Evaluation

The current implemenation supprts accessibility evaluation for any modes defined in settings.yml. Note that you can restrict the allowed uses (modes) on each link by adding a field of "allowed_uses" to link.csv following the example here. Othewise, links are open to all modes.

In order to perform multimodal accessibility evaluation, the corresponding modes (i.e., agent types) must be presented in settings.yml. It will be parsed by pyyaml (5.1 or higher) to the Python engine at run-time. Note that demand.csv is not necessary for accessibility evaluation.

agents:
  - type: p
    name: passenger
    vot: 10
    flow_type: 0
    pce: 1
    free_speed: 60
  - type: w
    name: walk
    vot: 10
    flow_type: 0
    pce: 1
    free_speed: 10

demand_periods:
  - period: AM
    time_period: 0700_0800

demand_files:
  - file_name: demand.csv
    format_type: column
    period: AM
    agent_type: p

If pyyaml is not installed or settings.yml is not provided, one demand period (AM) and one agent type (passenger) will be automatically created.

import path4gmns as pg

# no need to load demand file for accessibility evaluation
network = pg.read_network(load_demand=False)

print('\nstart accessibility evaluation\n')
st = time()

pg.evaluate_accessibility(network)

print('complete accessibility evaluation.\n')
print(f'processing time of accessibility evaluation: {time()-st:.2f} s')

Two formats of accessibility will be outputed: accessibility between each OD pair in terms of free flow travel time (accessibility.csv) and aggregated accessibility as to the number of accessible zones from each zone for each transportation mode specified in settings.yml given a budget time (up to 240 minutes) (accessibility_aggregated.csv). The following example is to evaluate accessibility only under the default mode (i.e., mode auto or agent type passenger).

import path4gmns as pg

# no need to load demand file for accessibility evaluation
network = pg.read_network(load_demand=False)

print('\nstart accessibility evaluation\n')
st = time()

pg.evaluate_accessibility(network, multimodal=False)
# the default is under mode auto (i.e., p)
# if you would like to evaluate accessibility under a target mode, say walk, then
# pg.evaluate_accessibility(network, multimodal=False, mode='w')
# or equivalently pg.evaluate_accessibility(network, multimodal=False, mode='walk')

print('complete accessibility evaluation.\n')
print(f'processing time of accessibility evaluation: {time()-st:.2f} s')

You can also get the accessible nodes and links within a time budget given a mode. Similar to the accessibility evalution, the selected mode must come from settings.yml.

import path4gmns as pg

# no need to load demand file for accessibility evaluation
network = pg.read_network(load_demand=False)

# get accessible nodes and links starting from node 1 with a 5-minitue
# time window for the default mode auto (i.e., 'p')
network.get_accessible_nodes(1, 5)
network.get_accessible_links(1, 5)

# get accessible nodes and links starting from node 1 with a 15-minitue
# time window for mode walk (i.e., 'w')
network.get_accessible_nodes(1, 15, 'w')
network.get_accessible_links(1, 15, 'w')
# the following two work equivalently as their counterparts above
# network.get_accessible_nodes(1, 15, 'walk')
# network.get_accessible_links(1, 15, 'walk')

When Time-Dependent Link Travel Time Matters

Link travel time is crucial in calculating accessibility. In the classic accessibility analysis, evalutaion networks are usually considered to be static in terms of link travel time, which is determined by link length and link free-flow speed under a specific mode. The free-flow speed comes from either link.csv or settings.yml (both are denoted as "free_speed"). When they are different, the minimum of these two will be adopted. The cases demonstrated above are all falling within this category.

Link travel time varies over time so does accessibility. When the time-dependent accessibility is of interested, time-dependent link travel time (i.e., VDF_fftt from a given demand period in link.csv) will come into play by overwriting the (static) link free-flow speed. This new feature is now part of v0.7.3 and is illustrated as below.

import path4gmns as pg

# no need to load demand file for accessibility evaluation
network = pg.read_network(load_demand=False)

print('\nstart accessibility evaluation\n')
st = time()

# time-dependent accessibility under the default mode auto (i.e., p)
# for demand period 0 (i.e., VDF_fftt1 in link.csv will be used in the evaluation)
pg.evaluate_accessibility(network, multimodal=False, time_dependent=True)

# if you would like to evaluate accessibility under a different demand period
# (say 1, which must be defined in settings.yml along with VDF_fftt2 in link.csv), then
pg.evaluate_accessibility(network, multimodal=False, time_dependent=True, demand_period_id=1)

# if you would like to evaluate accessibility under a target mode, say walk, then
pg.evaluate_accessibility(network, multimodal=False, mode='w', time_dependent=True)

While VDF_fftt begins with 1 (i.e., VDF_fftt1), the argument demand_period_id, corresponding to the sequence number of demand period appeared in demand_periods in settings.yml, starts from 0. So "demand_period_id=0" indicates that VDF_fftt1 will be used (and so on and so forth).

As VDF_fftt in link.csv can only accommodate one mode, time-dependent accessibility evaluation will require the user to prepare a mode-specific link.csv with dedicated VDF_fftt and allowed_uses. That's the reason that "multimodal=False" is always enforced in these examples.

Retrieve the time-dependent accessible nodes and links is similar to evaluate time-dependent accessibility by simply passing time_dependent and demand_period_id to get_accessible_nodes() and get_accessible_links().

import path4gmns as pg

# no need to load demand file for accessibility evaluation
network = pg.read_network(load_demand=False)

# get accessible nodes and links starting from node 1 with a 5-minitue
# time window for the default mode auto (i.e., 'p') for demand period 0
network.get_accessible_nodes(1, 5, time_dependent=True)

# get accessible nodes and links starting from node 1 with a 5-minitue
# time window for the default mode auto (i.e., 'p') for demand period 1 if it is defined
network.get_accessible_links(1, 5, time_dependent=True, demand_period_id=1)

# get accessible nodes and links starting from node 1 with a 15-minitue
# time window for mode walk (i.e., 'w') for demand periods 0 and 1 respectively
network.get_accessible_nodes(1, 15, 'w', time_dependent=True)
network.get_accessible_links(1, 15, 'w', time_dependent=True, demand_period_id=1)

Build Path4GMNS from Source

If you would like to test the latest features of Path4GMNS or have a compatible version to a specific operating system or an architecture, you can build the package from source and install it offline, where Python 3.x is required.

1. Build the Shared Libraries

The shared libraries of DTALite and path_engine for Path4GMNS can be built with a C++ compiler supporting C++11 and higher, where we use CMake to define the building process. Take path_engine for example,

# from the root directory of engine
$ mkdir build
$ cd build
$ cmake ..
$ cmake --build .

The last command can be replaced with $ make if your target system has Make installed. See here for details on how to build DTALite. After they are successfully compiled, move them to Path4GMSN/path4gmns/bin.

Caveat

As CMAKE_BUILD_TYPE will be IGNORED for IDE (Integrated Development Environment) generators, e.g., Visual Studio and Xcode, you will need to manually update the build type from debug to release in your IDE and build your target from there.

2. Build and Install the Python Package

# from the root directory of PATH4GMNS
$ python setup.py sdist bdist_wheel
$ cd dist
# or python -m pip instal pypath4gmns-0.7.3-py3-none-any.whl
$ python -m pip install path4gmns-0.7.3.tar.gz

Here, 0.7.3 is the version number. Replace it with the one specified in setup.py.

Benchmarks

Coming soon.

Upcoming Features

• Read and output node and link geometries (v0.6.0)
• Set up individual agents from aggregated OD demand only when it is needed (v0.6.0)
• Provide a setting file in yaml to let users control key parameters (v0.6.0)
• Support for multi-demand-period and multi-agent-type (v0.6.0)
• Load columns/paths from existing runs and continue path-base UE (v0.7.0a1)
• Download the predefined GMNS test data sets to usrs' local machines when needed (v0.7.0a1)
• Add allowed use in terms of agent type (i.e., transportation mode) for links (v0.7.0a1)
• Calculate and show up multimodal accessibilities (v0.7.0a1)
• Apply lightweight and faster implementation on accessibility evaluation using virtual centroids and connectors (v0.7.0)
• Get accessible nodes and links given mode and time budget (v0.7.0)
• Retrieve shortest paths under multimodal allowed uses (v0.7.2)
• Time-dependent accessibility evaluation (v0.7.3)
• Let users modify the network topology in a simple way by adding/removing nodes and links
• Enable manipulations on the overall travel demand and the demand between an OD pair
• Visualize individual column/paths on user's call
• Adopt parallel computing to further boost the performance

Contribute

Any contributions are welcomed including advise new applications of Path4GMNS, enhance documentation (this guideline and docstrings in the source code), refactor and/or optimize the source code, report and/or resolve potential issues/bugs, suggest and/or add new functionalities, etc.

Path4GMNS has a very simple workflow setup, i.e., master for release (on both GitHub and PyPI) and dev for development. If you would like to work directly on the source code (and probably the documentation), please make sure that the destination branch of your pull request is dev, i.e., all potential changes/updates shall go to the dev branch before merging into master for release.

You are encouraged to join our Slack workspace for more dicussions and collaborations. Drop us an email for invitation.

Implementation Notes

The column generation scheme in Path4GMNS is an equivalent single-processing implementation as its DTALite multiprocessing counterpart. Note that the results (i.e., column pool and trajectory for an agent) from Path4GMNS and DTALite are comparable but likely not identical as the shortest paths are usually not unique and subjected to implementations. This subtle difference should be gone and the link performances should be consistent if the iterations on both assignment and column generation are large enough. You can always compare the results (i.e., link_performance.csv) from Path4GMNS and DTALite given the same network and demand.

The whole package is implemented towards high performance. The core shortest-path engine is implemented in C++ (deque implementation of the modified label correcting algorithm) along with the equivalent Python implementations for demonstration. To achieve the maximum efficiency, we use a fixed-length array as the deque (rather than the STL deque) and combine the scan eligible list (represented as deque) with the node presence status. Along with the minimum and fast argument interfacing between the underlying C++ path engine and the upper Python modules, its running time is comparable to the pure C++-based DTALite for small- and medium-size networks (e.g., the Chicago Sketch Network). If you have an extremely large network and/or have requirement on CPU time, we recommend using DTALite to fully utilze its parallel computing feature.

An easy and smooth installation process by low dependency is one of our major design goals. The core Python modules in Path4GMNS only require a handful of components from the Python standard library (e.g., csv, cytpes, and so on) with no any third-party libraries/packages. On the C++ side, the precompiled path engine as shared libraries are embedded to make this package portable across three major desktop environments (i.e., Windows, macOS, and Linux) and its source is implemented in C++11 with no dependency. Users can easily build the path engine from the source code towards their target system if it is not listed above as one of the three.

References

Lu, C. C., Mahmassani, H. S., Zhou, X. (2009). Equivalent gap function-based reformulation and solution algorithm for the dynamic user equilibrium problem. Transportation Research Part B: Methodological, 43, 345-364.

Jayakrishnan, R., Tsai, W. K., Prashker, J. N., Rajadyaksha, S. (1994). A Faster Path-Based Algorithm for Traffic Assignment (Working Paper UCTC No. 191). The University of California Transportation Center.