netsy

NetSy (Network loads Synthesizer) is a general spatial resource-flow synthesizer. It turns case-specific activity, facility, vehicle, timing, network, and emissions evidence into explicit transactions, capacity-constrained movements, directional link loads, and emissions.

NetSy starts from actors and the transactions between them, observed or synthesized: observed shipments, provider-to-consumer matches, balanced OD flows, parcel demand, service trips, and facility-mediated chains all use the same transaction, tour, network, and emissions core.

Install

pip install netsy

Requires Python 3.11 or newer.

Visualizer

docs/ is a self-contained TypeScript visualizer for NetSy's CSV and GeoPackage files. It runs entirely in the browser: a map of the loaded geometries, the data-flow graph of every file NetSy reads and writes (where files are loaded), and their tables. No data is uploaded.

cd docs
npm install
npm run dev

The development server binds to the host network so it can be reviewed from another device on the local network. Its production output is static and is built with npm run build.

Usage

Run each command in the directory holding its input files.

netsy synth od-marginals             # location attributes + coefficients + OD prior -> productions, attractions
netsy synth od-flows                 # productions + attractions + OD prior -> od_flows.csv
netsy synth agents --seed 1          # zones + size totals + size classes -> agents.gpkg
netsy synth shipments --seed 1       # agents + zones + skims + shipment tables -> shipments.gpkg
netsy synth tours --seed 1           # shipments + zones + skims + tour tables -> tours.gpkg, tour_shipments.csv
netsy synth parcel-demand            # zones + depots + skims + parcel tables -> parcel_demand.csv
netsy synth deliveries --seed 1      # parcel demand + zones + skims -> delivery_tours.gpkg
netsy synth trip-matrices            # zones + skims + trip tables -> trip_matrices.omx
netsy synth network-loads            # tours, delivery tours, trip matrices + network -> network_loads.parquet
netsy synth network-emissions        # network loads + network + emission factors -> network_emissions.parquet
netsy synth facility-chains          # shipments + facilities + chain steps -> transaction_legs.gpkg

An existing output file is only replaced with --force. agents, shipments, tours, and deliveries make random draws; pass --seed to reproduce them.

Two options switch from MASS-GT's method to a contribution of NetSy's:

netsy synth deliveries --routing savings      # Clarke-Wright savings routes instead of MASS-GT's
netsy synth network-emissions --method copert # COPERT V instead of MASS-GT's emission factors

Reproducing MASS-GT

The Dutch case (built outside this repository, in its own directory with a build/ folder and run.sh) builds NetSy inputs from MASS-GT's own study-area data. Comparisons cover agents, SIF, shipments, freight tours, parcel demand/deliveries, van matrices, road assignment and emissions. Exact agreement is tested where appropriate; stochastic stages use independent seeds, and fixes are measured separately. The comparison audit, kept outside this repository with its scripts and results, reports the actual errors and differences, including the rounding-exact SIF and parcel-demand match. The Dutch adapter explicitly retains the source's reversed import/export reading and external-zone van mapping for reference compatibility; these are not general core rules.

Model boundary

NetSy deliberately separates the general model from the case-specific evidence that configures it.

This prevents a facility such as a UCC or microhub from becoming a special model branch. It is an ordinary facility in an explicit transaction chain. Whether a case selects that facility, and the parameters or rules used to do so, belong to scenario data rather than to the general core.

Data files

Agents

agents synthesizes the actors the way freight models such as MASS-GT synthesize firms: from each stratum's total size and a size distribution per agent class. A stratum is a zone and any other dimensions, such as an employment sector; it is also the agent class. size_classes.csv may be keyed by any subset of the stratum columns, so a size distribution can be national, per sector, per zone, or anything between. What an agent produces and consumes follows from its size and class later, when shipments are drawn.

For example, firms drawn from jobs per zone and sector with a national size distribution per sector:

--min-size drops agents smaller than a size once they are drawn, as MASS-GT drops firms of three employees or fewer; their size is not given to others.

agents.gpkg holds the actors: an agent_id, the stratum columns, size, and point geometry.

Shipments

shipments draws the shipments between agents, facilities and zones the way MASS-GT's shipment synthesis does, generalized to any resources, regions, flow types and facility types. Every row of shipment_totals.csv is a quantity of one shipment stratum to move from one region to another by one flow type. A shipment stratum is resource and any other columns, such as a logistic segment; a region is a group of zones, such as a study area or an external zone. A flow type names what each end is: an agent, a facility type, or a zone itself.

Key columns are any subset of the shipment strata (or, for rates, of the agent strata too), so a parameter can be global, per resource, per segment, or anything between.

For example, flow types of a freight model with distribution centres, external zones and parcel depots shared by operators:

Vehicle-choice terms are constant, transport_cost (the cost of the vehicles a shipment needs), size (its class quantity), size_class:<class>, origin:<kind> and destination:<kind> (the kind of an end), and distance_above:<km>. Delivery-choice terms are constant, log_duration, origin:<zone column> and destination:<zone column> (a numeric zone value at an end), and vehicle:<vehicle>.

shipments.gpkg has one shipment per row: transaction_id, the shipment strata, quantity, origin_agent_id and destination_agent_id (empty for an end that is not an agent), flow_type, the facility and zone of each end, size_class, vehicle, window, window_start and window_end, and a line from its origin to its destination.

facility-chains still counts quantities in whole units, so shipments of fractional quantities, such as tonnes, do not pass it yet.

Tours

tours forms vehicle tours from the shipments the way MASS-GT's tour formation does. Shipments are carried by carriers: a carrier based at each facility of the carrier types, and a number of independent carriers. Each carrier chains its shipments into tours with two end-tour logits.

End-tour terms are constant, duration and sqrt_duration (the tour's hours so far), utilization and squared_utilization (the share of its vehicle's capacity used), proximity (km to the nearest shipment left), log_stops, origin:<kind> and destination:<kind>, zone:<zone column> (the larger value at the two zones), vehicle:<vehicle> and resource:<resource> (the resource carried most). As in MASS-GT, the kinds, zones and vehicle are those of the tour's first shipment.

tours.gpkg has one trip per row: tour_id, trip (from 1), carrier, vehicle, resource (empty on the empty return), the other strata of the tour's first shipment, shipments, tour_quantity, quantity (on board), tour_departure, departure and arrival (hours), the trip's zones, and a line between them. tour_shipments.csv names the tour that carries each part of each shipment: transaction_id, tour_id, quantity.

Facility-mediated transactions

facility-chains is an optional general transshipment stage. It reads:

It writes transaction_legs.gpkg: one direct leg when no chain step is supplied, or an ordered set of legs when it is. Each facility consumes transaction quantity as throughput; an over-capacity scenario fails rather than silently routing through the facility.

For example, a package that must be consolidated at a UCC has one step:

and becomes supplier → ucc_1 followed by ucc_1 → customer. A microhub is represented the same way. Multiple ordered facilities produce multiple legs. Facility-choice, carrier-choice, parcel-scheduling, and service-demand models can create these transaction and chain inputs without changing the core chain semantics.

Parcels

parcel-demand derives parcels per zone the way MASS-GT's parcel demand module does, and deliveries forms delivery tours from depots the way its parcel scheduling does.

parcel_demand.csv has a row per destination zone, operator and depot with its parcels; delivery_tours.gpkg has a trip per row, in the shape of tours.gpkg, with the parcels delivered at its destination.

A consumer cell's utility is the sum of its matching terms; the probability of ordering at most a level is 1 / (1 + exp(utility − threshold)), and its expected parcels per period, times its persons over the period, are scaled to per_person parcels a day. Each zone's parcels are split by operator, rounded, and sent from the operator's nearest depot. With MASS-GT's routing a depot sends full vehicles first, then clusters its other parcels around the farthest left, orders each cluster nearest first and swaps pairs of stops. MASS-GT's code orders stops by the distances of the next zones in its list, an indexing error that makes its tours 27% longer than its clusters need; NetSy uses the right zones.

Contribution: --routing savings builds each depot's tours with the Clarke-Wright savings algorithm under the same load, then moves single stops while that shortens a tour. With MASS-GT's Dutch parcels it delivers the same 228,707 parcels with 10% fewer vehicle-kilometres than MASS-GT's routing, using 4% more tours.

Van trips

trip-matrices derives van trips between zones the way MASS-GT's service module does: per segment, productions (and as many attractions) linear in zone attributes, a gravity prior 1 / (1 + e^alpha cost^beta) times both zones' productions, and FRATAR balancing (25 iterations, 0.005 tolerance). Trips between two zones flagged external in zones.gpkg are dropped. It reads trip_attributes.csv (zone_id and attributes), trip_rates.csv (segment, attribute, rate) and trip_decay.csv (segment, alpha, beta, cost_per_hour, cost_per_km), and writes trip_matrices.omx, a matrix per segment.

Network loads

network-loads loads trips onto the road network the way MASS-GT's traffic assignment does: every trip takes the cheapest path between its zones' nodes, all or nothing, at the speeds it is given. It does not change speeds: where a case has the whole traffic of its roads, a separate step may derive speeds from it.

network_loads.parquet has a row per link, interval, vehicle and segment: trips, their mean utilization, and the link's speed for the vehicle.

Network emissions

network-emissions computes grams per load row and pollutant, a column per pollutant.

• --method factors (MASS-GT's) reads emission_factors.csv (vehicle, road_type, pollutant, empty, full in g/km); a load's factor lies between empty and full by its utilization. MASS-GT overwrites rather than adds parcel vans' emissions on urban links, which undercounts them; NetSy adds them.
• --method copert (a contribution) reads copert_vehicles.csv (vehicle, vehicle_type), copert_v_coefficients.csv (vehicle_type, pollutant, gradient_bin, payload_bin, alpha to eta, rf as a fraction, optional min_speed and max_speed) and non_exhaust_factors.csv (vehicle_type, pollutant, emission_factor in g/km). Hot exhaust follows COPERT V's speed function at each load's speed, its link's grade and its payload, snapped to COPERT's bins; a speed outside a row's range counts as the nearest bound. Non-exhaust emissions are flat factors, reported as <pollutant>_non_exhaust.

Canonical extension path

Actors and transactions can come from any source. NetSy's general core is designed to accept explicitly calibrated scenario adapters:

1. actor/facility population synthesis or observed registers;
2. production/attraction regressions, gravity, and FRATAR OD balancing;
3. shipment-size, vehicle, time, carrier, and facility-chain choices;
4. freight pickup/delivery, parcel, and service-operation tour strategies;
5. shortest-path, stochastic, and BPR assignment strategies;
6. COPERT V, legacy-factor, or other declared emissions-factor models;
7. stagewise calibration and validation against local OD, tours, counts, and emissions.

These adapters must declare their parameters, units, provenance, seed streams, and validation targets. They may reproduce a case study closely at aggregate/distributional levels, but NetSy does not claim row-for-row equivalence when a case uses a different tour heuristic, assignment model, emissions factors, or stochastic stream.

Balanced OD flows

od-marginals and od-flows derive a commodity matrix the way MASS-GT's spatial interaction freight module does: productions and attractions from linear regressions on location attributes, then FRATAR balancing of a base matrix to them. Both are independent of agents and write fractional flows that a case turns into shipment totals.

od-marginals writes productions.csv and attractions.csv (resource, location_id, quantity), which od-flows balances the prior to, writing od_flows.csv. Each FRATAR iteration scales every destination's column to its attraction and then every origin's row to its production, so the result matches the productions, and the attractions as closely as the prior allows. It stops when the largest scaling factor of either pass is within 0.005 of one (rounded to five decimals), when that measure stops changing, or after 50 iterations. Only supported pairs between listed locations carry flow; a marginal without support stays unmet, with a warning.

Model

Agents are drawn stratum by stratum, as MASS-GT draws firms. While a stratum $s$ has size $S_s$ left, an agent's size class $c$ is drawn with probability $p_{k(s),c}$ for its agent class $k(s)$. A closed class draws the size $e$ uniformly among the whole numbers from $\mathrm{min}_c$ to $\mathrm{max}_c$; an open class draws it, rounded, from a triangular distribution from $\mathrm{min}_c$ through a mode of $\max(\mathrm{min}_c, S_s/2)$ to $\max(\mathrm{median}_c, 1.5,S_s)$. A size above $1.3,S_s$ becomes $S_s$. $S_s$ falls by $e$, and the number of agents is where $S_s$ runs out. Agents of size zero, or below --min-size, are dropped, and each remaining agent is placed uniformly in its zone.

Shipments are drawn total by total, as MASS-GT draws them. While a total of stratum $s$ by flow type $f$ has quantity $Q$ left, one end is drawn by weight alone and the other by weight times distance decay: the destination first and the origin second, unless $f$ decays the destination. An agent's weight is its size times its rate for $s$, a facility's is its weight column, a zone's is one, and only candidates in the end's region (or pinned zone) count. The decay between origin zone $i$ and destination zone $j$ is $1 / (1 + e^{\alpha + \beta \ln c_{ij}})$, with $c_{ij} = \mathrm{cost/h} \cdot t_{ij} + \mathrm{cost/km} \cdot d_{ij}$; a zero skim cell, such as an intrazonal one, takes half the smallest positive value of its row. Ends that must share a facility column or differ are drawn among those that do. The size class $k$ and vehicle $v$ are then chosen together by multinomial logit, with $\mathrm{transport_cost} = (\mathrm{cost/h}_v \cdot t + \mathrm{cost/km}_v \cdot d) \lceil q_k / \mathrm{capacity}_v \rceil$; the shipment carries $\min(q_k, Q)$, and $Q$ falls by it. Last, each shipment's delivery window is chosen by multinomial logit over its stratum's windows.

With MASS-GT's Dutch inputs, regions are its superzones, flow types its twelve (external ones split by direction), and the shipment strata its NSTR commodity, logistic segment and flow-type id. Its exports read the commodity matrix's external-to-internal cells and its imports the reverse, a bug that the totals reproduce, as the reference run does. Two of its quirks are not reproduced: a parcel origin depot that equals the destination depot moves to its neighbour in MASS-GT's depot list, where NetSy draws among the operator's other depots, and its import draws reuse one random stream for every external zone.

Tours are formed carrier by carrier. Each shipment is split into equal parts that fit its vehicle; a part with an end at a facility of a carrier type goes to that facility's carrier (one of the two at random if both are), and any other to a random independent carrier. A based carrier takes its parts shortest first, an independent one in random order. A tour starts with the first part left. While it has fewer than max_shipments parts after its first, lasts less than max_duration, is less than full, and has a part left within max_proximity, a binary logit decides whether it ends: end_tour_first.csv after its first part, end_tour_later.csv after each later one. If it goes on, it takes the nearest part (a part of its vehicle counts as half as far) of the same same_stratum values whose weight keeps the tour under capacity_slack times its capacity and whose delivery window neighbours the tour's. Its stops are ordered loading first and then unloading, nearest first, starting from the carrier's base; single stops are then moved where that shortens the tour without unloading before loading. A tour returns empty to its start. With one dwell time per tour, drawn uniformly up to twice mean_dwell_time, its departure time puts its trips' arrivals, on average, midway between their windows' bounds; as in MASS-GT, the window of a stop is compared with the arrival at the stop after it.

With MASS-GT's Dutch inputs, carriers are based at distribution centres, tours keep to one logistic segment, and the transshipment and urban indicators are zone columns. Not reproduced: MASS-GT's transshipment indicator leaves out an end at a distribution centre in a terminal zone, and it drops the DC-loading indicator of a shipment unloaded in a terminal zone; parcel shipments of nominal flow types 3, 5 and 7 count as loaded at a distribution centre there.

Network loads give every trip its cheapest path at the given speeds, all or nothing, and sum trips and utilization per link, interval, vehicle and segment. Emissions are either MASS-GT's factors, interpolated between empty and full by utilization, or COPERT V's speed function at each load's speed, grade and payload, both times the link's length and trips.

Facility routing and powertrains

netsy synth facility-routing --seed 1 reads shipments.gpkg, zones, facilities, routing_settings.csv (zone_column, facility_column, area_column), routing_probabilities.csv (shipment strata and probability), and routing_vehicles.csv (shipment strata, vehicle, share). It writes routed_shipments.gpkg: unique transaction/leg IDs, stable parent_transaction_id, ordered leg, and an explicit carrier for facility-to-policy-area movements. Quantities may be fractional. Each leg retains the consignment quantity; summing all legs counts transport work, not new final demand.

netsy synth facility-routing --seed 1
netsy synth tours --shipments routed_shipments.gpkg --seed 1

Tours honor explicit carrier assignments. Optional combustion_shares.csv contains choice keys (e.g. carrier_type, zone_relation, segment, vehicle), combustion, and share; empty keys are wildcards and more specific keys win. zone_relation is all, some, or none of the tour's actual zones flagged by the zone_column in combustion_settings.csv. Unspecified vehicle/segment fallbacks must be supplied as scenario rows, never an implicit uniform fuel draw.

netsy synth parcel-routing --seed 1 reads parcel_demand.csv, zones, facilities, skims, parcel_routing_settings.csv and parcel_routing_vehicles.csv, and writes routed_parcel_demand.csv. Settings select the flag and threshold, facility type, consolidation probability, and nearest or minimum_detour selection. Optional operator_column restricts hubs to their operator; area_column restricts them to the recipient's area. pool_operators=1 consolidates outbound operations while preserving demand_operator; inbound_vehicle selects the feeder vehicle. Vehicle shares can have maximum_distance (km) and fallback_vehicle for a radius-limited mode. final_delivery distinguishes customer parcels from transfers.

netsy synth parcel-routing --seed 1
netsy synth deliveries --demand routed_parcel_demand.csv --seed 1

Optional delivery_vehicles.csv supplies vehicle, max_load (parcels), drop_time (hours/parcel), speed_multiplier relative to the time skim (or absolute speed_kmh), and utilization. Tours are grouped by facility, operator, vehicle and leg. Optional delivery_combustion_shares.csv uses the same tour-choice contract. Case adapters, including the Dutch UCC and microhub scenarios, live with their case data outside this repository.

Assignment preserves combustion in its output strata. road_costs.csv can add a combustion selector, an attribute/value filter on network links, and operation=add or exclude (otherwise replace), allowing hybrid route policies without country-specific branches. Matrix assignment can use combustion_zone_column and flagged_combustion to classify by real endpoint flags. Optional from_node_order/to_node_order preserve an input model's shortest-path tie ordering. assignment_settings.csv can set route_samples, seed and jitter for MASS-GT-style perturbed-cost route averaging; the default remains one route. NETSY_WORKERS bounds worker processes (default at most four).

emission_multipliers.csv optionally supplies multiplier keyed by combustion, vehicle, pollutant, source (exhaust or non_exhaust), or link attributes. Specific rules override wildcard rules; overlapping equal-specificity rules fail. Zero tailpipe factors never automatically zero tire/brake emissions.

For COPERT use netsy synth network-loads --payload-strata first: this keeps separate 0/50/100 percent payload bins. Averaging payload before evaluating a nonlinear emission function is generally not equivalent. Speeds still come from the supplied link speed tables. Missing COPERT vehicle mappings or coefficient coverage fail explicitly. The Dutch COPERT example declares an assumed Euro VI truck / Euro 6 van fleet and flat grades; it is a factor-model sensitivity, not an observed fleet estimate or a reproduction of the advisor paper's case. The current Dutch non-exhaust input is empty: no non-exhaust result is claimed. A battery-related non-exhaust sensitivity can be supplied through explicit scenario multipliers; it is not inferred from an electric label.

Comparisons and presentation

The comparison audit (reference revisions, execution repairs, seed sensitivity, stage-level comparisons and reproducible commands) and the presentation summarizing it are kept outside this repository.

netsy synth indicators writes tidy indicators.csv: counts, quantities, customer versus transfer parcels, vehicle-km, and pollutant grams, plus OD/van matrix totals. It replaces country-specific reporting labels with optional indicator_groups.csv rows (table, column; empty column means a total). Group a table by any of its attributes. quantity_units and size_units retain case-defined units; the Dutch adapter uses tonnes and jobs. Vehicle-km always uses explicit road-link lengths, rather than straight-line tour geometry.