sensor-routing 0.2.3

Optimal routing for CRNS mobile sensor data collection

Sensor Routing

Optimal routing solution for mobile Cosmic Ray Neutron Sensing (CRNS) data collection. This package provides sophisticated algorithms for calculating efficient routes that maximize information value while minimizing travel distance and time.

Features

• 🗺️ Geospatial Route Optimization: Calculate optimal routes using real-world road networks from OpenStreetMap
• 📊 Information Value Maximization: Balance between spatial coverage and information gain
• 🔄 Multiple Routing Strategies: Support for both standard and economical routing approaches
• 🎯 Point Mapping: Map sensor locations to road networks with advanced filtering
• 📈 Benefit Calculation: Evaluate information value of different route segments
• 🛣️ Path Finding: Dijkstra-based algorithms with custom cost functions
• 🔍 Hull Point Extraction: Optimize sensor placement using convex hull analysis

Installation

From PyPI (recommended)

pip install sensor-routing

From source

git clone https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing.git
cd sensor-routing
pip install -e .

Development installation

pip install -e ".[dev]"

Quick Start

Command Line Interface

The package provides a command-line interface for the full pipeline:

sensor-routing --wd /path/to/work_directory

Python API

from sensor_routing import point_mapping, benefit_calculation, path_finding, route_finding

# Map points to road network
pm_output = point_mapping.point_mapping(
    points_path="input/points.csv",
    osm_path="input/osm_data.geojson",
    output_path="output"
)

# Calculate benefits
bc_output = benefit_calculation.benefit_calculation(
    pm_output=pm_output,
    output_path="output"
)

# Find optimal path
pf_output = path_finding.path_finding(
    bc_output=bc_output,
    output_path="output"
)

# Generate final route
route = route_finding.route_finding(
    pf_output=pf_output,
    output_path="output"
)

Requirements

• Python 3.12 or higher
• See requirements.txt for full dependency list

Key Dependencies

• NumPy & Pandas: Numerical and data processing
• GeoPandas: Geospatial data handling
• OSMnx: OpenStreetMap network analysis
• NetworkX: Graph-based routing algorithms
• Shapely: Geometric operations
• SciPy & scikit-learn: Scientific computing and machine learning
• h5py: MATLAB v7.3 HDF5 file support
• Pydantic: Data validation

Project Structure

sensor_routing/
├── point_mapping.py          # Map sensor points to road network
├── benefit_calculation.py    # Calculate information value
├── path_finding.py           # Find optimal paths
├── route_finding.py          # Generate final routes
├── hull_points_extraction.py # Extract convex hull points
├── econ_mapping.py           # Economic point mapping variant
├── econ_benefit.py           # Economic benefit calculation variant
├── econ_paths.py             # Economic path finding variant
├── econ_route.py             # Economic route finding variant
└── full_pipeline_cli.py      # Command-line interface

Usage

Working Directory Structure

The pipeline expects a working directory with the following structure:

work_dir/
├── input/
│   ├── converted.csv         # Sensor point locations (EPSG:25832)
│   ├── osm_data.geojson      # OpenStreetMap road network
│   ├── predictors.txt        # Environmental predictors (auto-generated if missing)
│   └── *.mat                 # MATLAB files for predictor generation
├── transient/                # Intermediate pipeline outputs
└── debug/                    # Debug outputs (optional, if DEBUG=True)

Input Data Format

Road Network

osm_data.geojson: GeoJSON file containing road network from OpenStreetMap

Environmental Predictors

predictors.txt: Space-separated file with environmental variables (auto-generated from MATLAB files if not present):

X               Y               Mask            Predictor1      Predictor2      ...
6.1950000e+05   5.7865000e+06   0.0000000e+00   1.3295830e+02   2.2212509e+02   ...
6.1950000e+05   5.7862500e+06   0.0000000e+00   1.3180805e+02   2.1562871e+02   ...

Format Requirements:

• Scientific notation (e.g., 6.1950000e+05)
• Column 1: X coordinate (Easting)
• Column 2: Y coordinate (Northing)
• Column 3: Urban mask (0=rural, 1=urban/NaN in predictors)
• Columns 4+: Environmental predictor values

MATLAB File Support (Auto-conversion)

The pipeline can automatically generate predictors.txt from MATLAB files:

Supported MATLAB formats:

• .mat files (< v7.3): Read with scipy.io
• .mat files (v7.3 HDF5): Read with h5py

File naming convention:

• Predictor name is extracted from the first part of the filename (before first _ or .)
• Rest of filename can be any format (metadata, version, EPSG code, etc.)

Examples:

input/
├── Predictor1.mat           # Predictor: Predictor1
├── Predictor2_metadata.mat  # Predictor: Predictor2
├── Predictor3.mat           # Predictor: Predictor3
├── Temperature.mat          # Predictor: Temperature
└── Moisture_v2.mat          # Predictor: Moisture

Requirements:

• Predictor name is extracted from the first part of filename (before first _)
• Each .mat file should contain:
  • map: Predictor values (2D array, will be flattened)
  • outX: X coordinates (Easting)
  • outY: Y coordinates (Northing)
• All files must have the same number of data points
• NaN values in any predictor automatically mark that point as urban (mask=1)

Auto-generation behavior:

1. Pipeline checks for predictors.txt in input/ directory
2. If not found, searches for .mat files
3. Merges all MATLAB files into single predictors.txt
4. Columns ordered as: X, Y, Mask, <alphabetically sorted predictors>
5. Continues with routing pipeline

Pipeline Parameters

The pipeline can be configured via full_pipeline_parameters.json:

{
    "CRS": "EPSG:25832",
    "EPSG": 25832,
    "information_weight": 0.5,
    "start_node": null,
    "end_node": null,
    "max_iterations": 100,
    "enable_module_debug": false
}

Debug Mode

Enable debug output by setting ENABLE_MODULE_DEBUG = True in full_pipeline_cli.py or via parameters file. This will:

• Print detailed progress information
• Save intermediate results to debug/ directory
• Show progress bars for long-running operations

Development

Running Tests

pytest test/

Code Formatting

black sensor_routing/
flake8 sensor_routing/

Type Checking

mypy sensor_routing/

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Merge Request

Documentation

For detailed documentation on specific modules:

• Point Mapping: See HOW_TO_USE_FOR_ROUTING.md
• Benefit Calculation: See IMPROVED_INFORMATION_VALUE_EXPLANATION.md
• Debug Control: See DEBUG_CONTROL_GUIDE.md
• Information Weights: See INFORMATION_WEIGHT_RANGES.md

Citation

If you use this software in your research, please cite:

@software{sensor_routing,
  author = {Topaclioglu, Can},
  title = {Sensor Routing: Optimal routing for CRNS mobile sensor data collection},
  year = {2024},
  url = {https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing}
}

License

This project is licensed under the European Union Public License 1.2 (EUPL-1.2). See the LICENSE file for details.

Authors

• Can Topaclioglu - Initial work - UFZ

Acknowledgments

• Helmholtz Centre for Environmental Research (UFZ)
• Department of Monitoring and Exploration Technologies

Support

For questions, issues, or feature requests:

• Open an issue on GitLab
• Contact: [e-mail]

Changelog

Version 0.2.2 (Current)

• ✨ Added automatic MATLAB .mat file to predictors.txt conversion
• ✨ Support for both old (<v7.3) and new (v7.3 HDF5) MATLAB formats
• ✨ Automatic urban mask generation from NaN values in predictors
• ✨ Added h5py dependency for MATLAB v7.3 support
• 📦 Updated dependencies for PyPI distribution
• 🐛 Fixed hull_points_extraction summary_kwargs bug
• 📝 Enhanced documentation with MATLAB file requirements

Version 0.2.1

• ✨ Added comprehensive debug control system
• ✨ Migrated to Pydantic V2
• ✨ Added economic routing variants
• 🐛 Fixed multiple debug output issues
• 📦 Prepared for PyPI distribution
• 📝 Improved documentation

Version 0.1.15

• Initial release with basic routing functionality