Metadata-Version: 2.5
Name: gri-tropo-data
Version: 0.2.4
Summary: ERA5 data pipeline for generating tropospheric refractivity datasets
Project-URL: Homepage, https://geosolresearch.com
Project-URL: Repository, https://gitlab.com/geosol-foss/python/gri-tropo-data
Project-URL: Issues, https://gitlab.com/geosol-foss/python/gri-tropo-data/-/issues
Project-URL: Changelog, https://gitlab.com/geosol-foss/python/gri-tropo-data/-/releases
Author-email: GeoSol Research Inc <contact@geosolresearch.com>
License-Expression: MIT
License-File: LICENSE
Keywords: ERA5,data-pipeline,refractivity,troposphere
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
Requires-Python: >=3.12
Requires-Dist: ecmwf-datastores-client>=0.4.1
Requires-Dist: gri-utils>=0.3.3
Requires-Dist: netcdf4>=1.7.3
Requires-Dist: numpy>=2.3.3
Requires-Dist: xarray>=2025.11.0
Description-Content-Type: text/markdown

[![GeoSol Research Logo](https://geosolresearch.com/logos/foss_logo.png "GeoSol Research")](https://geosolresearch.com)

# Tropo Data (Troposphere Data Processing)

ERA5 data download and processing tools for generating tropospheric refractivity datasets used by [gri-tropo](https://gitlab.com/geosol-foss/python/gri-tropo). Requires Python 3.12+.

## Overview

This package provides scripts for:

- **Downloading ERA5 reanalysis data** from the Copernicus Climate Data Store (CDS)
- **Processing surface data** to calculate surface refractivity (n_sur)
- **Processing pressure-level data** to calculate reference heights (ref_ht)

The output NPZ files are consumed by the `gri-tropo` library for tropospheric correction calculations.

## Documentation

The wheel ships its documentation inside the package, in `gri_tropo_data/docs/`,
so it is available wherever the package is installed:

- `overview.md` -- the download and update scripts, the refractivity and
  scale-height math, and the file format
- `api_summary.md` -- every public function and signature (generated)

Print the directory with
`python -c "import gri_tropo_data, pathlib; print(pathlib.Path(gri_tropo_data.__file__).parent / 'docs')"`.
Every example in those files is run by the test suite.

## Installation

```bash
# Clone and setup
git clone https://gitlab.com/geosol-foss/python/gri-tropo-data.git
cd gri-tropo-data
uv sync

# Activate environment
source .venv/bin/activate
```

## Usage

### 1. Download ERA5 Data

Download ERA5 surface and pressure-level data for a specific date:

```bash
python -m gri_tropo_data.download_era5 2025 03 15
```

**Output:**

- `<era5_raw>/era5_surface_2025_03_15.nc` - Surface meteorological data
- `<era5_raw>/era5_pressure_2025_03_15.nc` - Pressure-level data

**Note:** Requires a [Copernicus Climate Data Store](https://cds.climate.copernicus.eu/) account with the ERA5 licences accepted, and an API key in `~/.ecmwfdatastoresrc`. `python -m gri_tropo_data.download_era5 --help` prints the setup steps.

### 2. Process Surface Refractivity Data

Process all unprocessed ERA5 surface data to generate n_sur files:

```bash
python -m gri_tropo_data.update_nsur
```

**Output:**

- `<n_sur>/n_sur_2025_03_15.npz` - Surface refractivity data

### 3. Process Reference Height Data

Process all unprocessed ERA5 pressure-level data to generate ref_ht files:

```bash
python -m gri_tropo_data.update_refht
```

**Output:**

- `<ref_ht>/ref_ht_2025_03_15.npz` - Reference height data

## Data Formats

### Downloaded ERA5 Data

**Surface Data (Single-Level):**

- Variables: 2m temperature, surface pressure, 2m dewpoint temperature
- Temporal resolution: Hourly (24 snapshots per day)
- Spatial resolution: 1 x 1 degree global grid

**Pressure-Level Data:**

- Variables: Temperature, geopotential, specific humidity
- Pressure levels: 1000, 925, 850, 700, 500, 300 hPa
- Temporal resolution: Hourly (24 snapshots per day)
- Spatial resolution: 1 x 1 degree global grid

### Processed NPZ Files

**n_sur files** - Shape: `(181, 360, 24)` for `(latitude, longitude, hour)`

- Surface refractivity in N-units
- Hourly data for a single day

**ref_ht files** - Shape: `(181, 360)` for `(latitude, longitude)`

- Reference height in meters (scale height of exponential refractivity decay)
- Daily average data

**Indexing scheme:**

- Latitudes: indices 0-90 for 0 to 90 degrees N, indices 91-180 for -90 to -1 degrees S
- Longitudes: indices 0-359 for 0 to 359 degrees E
- Hours (n_sur only): indices 0-23 for UTC hours

The full format, with executed examples, is in `gri_tropo_data/docs/overview.md`.

## Configuration

The directories come from the gri-utils config file, `~/.config/gri/config.toml`,
falling back to the paths shown:

```toml
[gri-tropo.era5_raw]
data_dir = "/data/work/tropo/era5_raw"
[gri-tropo.n_sur]
data_dir = "/data/work/tropo/n_sur"
[gri-tropo.ref_ht]
data_dir = "/data/work/tropo/ref_ht"
```

`update_nsur` and `update_refht` take an optional output format (`npz`, the
default, `csv`, or `both`) and process every downloaded day that does not yet
have an output in that format.


## Other Projects

Current list of other [GRI FOSS Projects](https://gitlab.com/geosol-foss/python/gri-tropo-data/-/blob/main/.docs_other_projects.md) we are building and maintaining.

## License

MIT License. See [LICENSE](https://gitlab.com/geosol-foss/python/gri-tropo-data/-/blob/main/LICENSE) for details.
