Metadata-Version: 2.4
Name: sensor-routing
Version: 0.2.3
Summary: Optimal routing for CRNS mobile sensor data collection
Home-page: https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing
Author: Can Topaclioglu
Author-email: Can Topaclioglu <can.topaclioglu@ufz.de>
Maintainer-email: Can Topaclioglu <can.topaclioglu@ufz.de>
License: EUPL-1.2
Project-URL: Homepage, https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing
Project-URL: Repository, https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing
Project-URL: Issues, https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing/-/issues
Keywords: sensor-routing,CRNS,cosmic-ray-neutron-sensing,geospatial,routing-optimization,network-analysis,path-finding
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: European Union Public Licence 1.2 (EUPL 1.2)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: GIS
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.2.0
Requires-Dist: pandas>=2.2.3
Requires-Dist: geopandas>=1.0.1
Requires-Dist: osmnx>=2.0.0
Requires-Dist: shapely>=2.0.6
Requires-Dist: pyproj>=3.7.0
Requires-Dist: pyogrio>=0.10.0
Requires-Dist: networkx>=3.4.2
Requires-Dist: scipy>=1.11.0
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: annotated-types>=0.6.0
Requires-Dist: tqdm>=4.66.0
Requires-Dist: requests>=2.32.3
Requires-Dist: h5py>=3.8.0
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: mypy>=1.5.0; extra == "dev"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Sensor Routing

[![Python Version](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-EUPL--1.2-green.svg)](https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12)

Optimal routing solution for mobile Cosmic Ray Neutron Sensing (CRNS) data collection. This package provides sophisticated algorithms for calculating efficient routes that maximize information value while minimizing travel distance and time.

## Features

- 🗺️ **Geospatial Route Optimization**: Calculate optimal routes using real-world road networks from OpenStreetMap
- 📊 **Information Value Maximization**: Balance between spatial coverage and information gain
- 🔄 **Multiple Routing Strategies**: Support for both standard and economical routing approaches
- 🎯 **Point Mapping**: Map sensor locations to road networks with advanced filtering
- 📈 **Benefit Calculation**: Evaluate information value of different route segments
- 🛣️ **Path Finding**: Dijkstra-based algorithms with custom cost functions
- 🔍 **Hull Point Extraction**: Optimize sensor placement using convex hull analysis

## Installation

### From PyPI (recommended)

```bash
pip install sensor-routing
```

### From source

```bash
git clone https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing.git
cd sensor-routing
pip install -e .
```

### Development installation

```bash
pip install -e ".[dev]"
```

## Quick Start

### Command Line Interface

The package provides a command-line interface for the full pipeline:

```bash
sensor-routing --wd /path/to/work_directory
```

### Python API

```python
from sensor_routing import point_mapping, benefit_calculation, path_finding, route_finding

# Map points to road network
pm_output = point_mapping.point_mapping(
    points_path="input/points.csv",
    osm_path="input/osm_data.geojson",
    output_path="output"
)

# Calculate benefits
bc_output = benefit_calculation.benefit_calculation(
    pm_output=pm_output,
    output_path="output"
)

# Find optimal path
pf_output = path_finding.path_finding(
    bc_output=bc_output,
    output_path="output"
)

# Generate final route
route = route_finding.route_finding(
    pf_output=pf_output,
    output_path="output"
)
```

## Requirements

- Python 3.12 or higher
- See `requirements.txt` for full dependency list

### Key Dependencies

- **NumPy** & **Pandas**: Numerical and data processing
- **GeoPandas**: Geospatial data handling
- **OSMnx**: OpenStreetMap network analysis
- **NetworkX**: Graph-based routing algorithms
- **Shapely**: Geometric operations
- **SciPy** & **scikit-learn**: Scientific computing and machine learning
- **h5py**: MATLAB v7.3 HDF5 file support
- **Pydantic**: Data validation

## Project Structure

```
sensor_routing/
├── point_mapping.py          # Map sensor points to road network
├── benefit_calculation.py    # Calculate information value
├── path_finding.py           # Find optimal paths
├── route_finding.py          # Generate final routes
├── hull_points_extraction.py # Extract convex hull points
├── econ_mapping.py           # Economic point mapping variant
├── econ_benefit.py           # Economic benefit calculation variant
├── econ_paths.py             # Economic path finding variant
├── econ_route.py             # Economic route finding variant
└── full_pipeline_cli.py      # Command-line interface
```

## Usage

### Working Directory Structure

The pipeline expects a working directory with the following structure:

```
work_dir/
├── input/
│   ├── converted.csv         # Sensor point locations (EPSG:25832)
│   ├── osm_data.geojson      # OpenStreetMap road network
│   ├── predictors.txt        # Environmental predictors (auto-generated if missing)
│   └── *.mat                 # MATLAB files for predictor generation
├── transient/                # Intermediate pipeline outputs
└── debug/                    # Debug outputs (optional, if DEBUG=True)
```

### Input Data Format

#### Road Network
**osm_data.geojson**: GeoJSON file containing road network from OpenStreetMap

#### Environmental Predictors
**predictors.txt**: Space-separated file with environmental variables (auto-generated from MATLAB files if not present):
```
X               Y               Mask            Predictor1      Predictor2      ...
6.1950000e+05   5.7865000e+06   0.0000000e+00   1.3295830e+02   2.2212509e+02   ...
6.1950000e+05   5.7862500e+06   0.0000000e+00   1.3180805e+02   2.1562871e+02   ...
```

**Format Requirements:**
- Scientific notation (e.g., `6.1950000e+05`)
- Column 1: X coordinate (Easting)
- Column 2: Y coordinate (Northing)
- Column 3: Urban mask (0=rural, 1=urban/NaN in predictors)
- Columns 4+: Environmental predictor values

#### MATLAB File Support (Auto-conversion)
The pipeline can automatically generate `predictors.txt` from MATLAB files:

**Supported MATLAB formats:**
- `.mat` files (< v7.3): Read with scipy.io
- `.mat` files (v7.3 HDF5): Read with h5py

**File naming convention:**
- Predictor name is extracted from the **first part** of the filename (before first `_` or `.`)
- Rest of filename can be any format (metadata, version, EPSG code, etc.)

**Examples:**
```
input/
├── Predictor1.mat           # Predictor: Predictor1
├── Predictor2_metadata.mat  # Predictor: Predictor2
├── Predictor3.mat           # Predictor: Predictor3
├── Temperature.mat          # Predictor: Temperature
└── Moisture_v2.mat          # Predictor: Moisture
```

**Requirements:**
- Predictor name is extracted from the first part of filename (before first `_`)
- Each `.mat` file should contain:
  - `map`: Predictor values (2D array, will be flattened)
  - `outX`: X coordinates (Easting)
  - `outY`: Y coordinates (Northing)
- All files must have the same number of data points
- NaN values in any predictor automatically mark that point as urban (mask=1)

**Auto-generation behavior:**
1. Pipeline checks for `predictors.txt` in `input/` directory
2. If not found, searches for `.mat` files
3. Merges all MATLAB files into single `predictors.txt`
4. Columns ordered as: `X, Y, Mask, <alphabetically sorted predictors>`
5. Continues with routing pipeline

### Pipeline Parameters

The pipeline can be configured via `full_pipeline_parameters.json`:

```json
{
    "CRS": "EPSG:25832",
    "EPSG": 25832,
    "information_weight": 0.5,
    "start_node": null,
    "end_node": null,
    "max_iterations": 100,
    "enable_module_debug": false
}
```

### Debug Mode

Enable debug output by setting `ENABLE_MODULE_DEBUG = True` in `full_pipeline_cli.py` or via parameters file. This will:
- Print detailed progress information
- Save intermediate results to `debug/` directory
- Show progress bars for long-running operations

## Development

### Running Tests

```bash
pytest test/
```

### Code Formatting

```bash
black sensor_routing/
flake8 sensor_routing/
```

### Type Checking

```bash
mypy sensor_routing/
```

## Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Merge Request

## Documentation

For detailed documentation on specific modules:

- **Point Mapping**: See `HOW_TO_USE_FOR_ROUTING.md`
- **Benefit Calculation**: See `IMPROVED_INFORMATION_VALUE_EXPLANATION.md`
- **Debug Control**: See `DEBUG_CONTROL_GUIDE.md`
- **Information Weights**: See `INFORMATION_WEIGHT_RANGES.md`

## Citation

If you use this software in your research, please cite:

```bibtex
@software{sensor_routing,
  author = {Topaclioglu, Can},
  title = {Sensor Routing: Optimal routing for CRNS mobile sensor data collection},
  year = {2024},
  url = {https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing}
}
```

## License

This project is licensed under the European Union Public License 1.2 (EUPL-1.2). See the [LICENSE](LICENSE) file for details.

## Authors

- **Can Topaclioglu** - *Initial work* - [UFZ](https://www.ufz.de/)

## Acknowledgments

- Helmholtz Centre for Environmental Research (UFZ)
- Department of Monitoring and Exploration Technologies

## Support

For questions, issues, or feature requests:
- Open an issue on [GitLab](https://codebase.helmholtz.cloud/ufz/tb5-smm/met/wg7/sensor-routing/-/issues)
- Contact: can.topaclioglu@ufz.de

## Changelog

### Version 0.2.2 (Current)
- ✨ Added automatic MATLAB .mat file to predictors.txt conversion
- ✨ Support for both old (<v7.3) and new (v7.3 HDF5) MATLAB formats
- ✨ Automatic urban mask generation from NaN values in predictors
- ✨ Added h5py dependency for MATLAB v7.3 support
- 📦 Updated dependencies for PyPI distribution
- 🐛 Fixed hull_points_extraction summary_kwargs bug
- 📝 Enhanced documentation with MATLAB file requirements

### Version 0.2.1
- ✨ Added comprehensive debug control system
- ✨ Migrated to Pydantic V2
- ✨ Added economic routing variants
- 🐛 Fixed multiple debug output issues
- 📦 Prepared for PyPI distribution
- 📝 Improved documentation

### Version 0.1.15
- Initial release with basic routing functionality
