Metadata-Version: 2.4
Name: qpkit
Version: 0.1.0
Summary: NOAA precipitation GRIB2 downloader and DSS writer for MRMS QPE, WPC QPF, and HRRR
Author: WEST Consultants, Inc.
Author-email: Gyan Basyal <gyanbasyalz@gmail.com>
License-Expression: Apache-2.0
Keywords: noaa,grib2,qpe,qpf,hrrr,mrms,dss,hydrology
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Environment :: Console
Classifier: Topic :: Scientific/Engineering :: Hydrology
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.6
Requires-Dist: pydsstools>=3.1.0
Provides-Extra: dev
Requires-Dist: myst-parser>=2.0; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Requires-Dist: sphinx>=9.0; extra == "dev"
Requires-Dist: sphinx-autodoc-typehints>=2.0; extra == "dev"
Requires-Dist: mypy>=1.8; extra == "dev"
Dynamic: license-file

# qpkit

qpkit is a Python library for downloading NOAA precipitation GRIB2 products —
MRMS QPE, WPC QPF, and HRRR — and, for MRMS QPE and HRRR, writing the
downloaded grids directly into HEC-DSS files for use in hydrologic models.

## Installation

Clone the repository and install qpkit in editable mode:

```console
git clone https://github.com/gyanz/qpkit.git
cd qpkit
python -m pip install -e .
```

Editable mode requires a local clone, but if you just want to install the
latest code without one, pip can install directly from a Git URL:

```console
python -m pip install git+https://github.com/gyanz/qpkit.git
```

An optional `dev` extra adds developer tools (pytest, ruff, mypy, sphinx):

```console
python -m pip install -e ".[dev]"
```

## Products

`qpkit.QPKit` is the high-level entry point for every supported product. Use
it as a context manager so the underlying HTTP client closes cleanly:

```python
from qpkit import QPKit

with QPKit() as kit:
    ...
```

| Request        | Product family                                                              | Source                         | Writes to DSS |
|----------------|-----------------------------------------------------------------------------|--------------------------------|---------------|
| `QPERequest`   | MRMS QPE — `MultiSensor_Pass1`, `MultiSensor_Pass2`, `RadarOnly`, `MultiSensor_Pass1_Pass2` | MRMS (live) / AWS S3 (archive) | Yes — `QPEGridOptions` |
| `QPFRequest`   | WPC gridded QPF                                                             | WPC 2.5 km GRIB FTP            | Yes — `QPFGridOptions` |
| `HRRRRequest`  | HRRR — `APCP` (accumulated precip) or `PRATE` (precip rate)                | NOMADS GRIB filter             | Yes — `HRRRGridOptions` |

`download_qpe` / `download_qpf` / `download_hrrr` (and the generic `download`)
return a `DownloadResult`:

| Field      | Type                       | Meaning                                |
|------------|----------------------------|----------------------------------------|
| `succeeded`| `tuple[Path, ...]`         | Paths that were written.               |
| `failed`   | `tuple[str, ...]`          | URLs that did not download.            |
| `skipped`  | `tuple[Path, ...]`         | Paths to files already present locally.|
| `plan`     | `DownloadPlan`             | The plan that was executed.            |

`DownloadResult.all_files` returns a sorted `tuple[Path, ...]` combining
`succeeded` and `skipped` — convenient when feeding the full set of available
files into a downstream processing step.

Build a plan without downloading — `build_plan` dispatches on the request
type (`QPERequest`, `QPFRequest`, or `HRRRRequest`); product-specific
`build_plan_qpe` / `build_plan_qpf` / `build_plan_hrrr` are also available:

```python
plan = kit.build_plan(request, "download")
for item in plan.items:
    print(item.filename, item.url, item.valid_time)
```

Each `DownloadItem` exposes `filename`, `url`, and an optional `valid_time`
(populated from the embedded timestamp where the product encodes one). A
`DownloadPlan` carries the `output_dir` it was built for, so two-stage
execution is just `kit.downloader.download(plan)` — no second argument.

### QPE (MRMS)

Important `QPERequest` parameters:

- `product` — `MultiSensor_Pass1` (lower latency, less QA/QC),
  `MultiSensor_Pass2` (higher quality, more latency), `RadarOnly`
  (radar-derived, no gauge correction), or `MultiSensor_Pass1_Pass2` (hybrid:
  Pass1 for the most recent ~25 hours, Pass2 for everything older — requires
  `start`/`end`).
- `interval` — accumulation window in hours: `1, 3, 6, 12, 24, 48, 72`.
- `start` / `end` — optional timezone-aware datetimes for an archive time
  range (AWS S3, available from 2020-10-14 onward).
- `source` — `"auto"` (NCEP for live requests, AWS when a time range is given),
  `"aws"`, or `"ncep"`.

Download QPE GRIB2 files:

```python
from pathlib import Path
from qpkit import QPERequest, QPKit

with QPKit() as kit:
    request = QPERequest(product="MultiSensor_Pass1", interval=1)
    result = kit.download_qpe(request, Path("download"))

    print(f"downloaded={len(result.succeeded)} "
          f"skipped={len(result.skipped)} "
          f"failed={len(result.failed)}")
```

Download and write the grids to a DSS file in one step with
`download_to_dss` and `QPEGridOptions`:

```python
from pydsstools.core.crs import shg
from qpkit import BoundingBox, QPERequest, QPKit
from qpkit.models import QPEGridOptions

with QPKit() as kit:
    request = QPERequest(product="MultiSensor_Pass1", interval=1)
    options = QPEGridOptions(
        part_a="MRMS",
        part_b="SEATTLE",  # name the cropped region, not CONUS, when `extents` is set
        part_c="PRECIP",
        crs=shg(),  # destination CRS for reprojection; this is also the QPEGridOptions default
        cell_size=1000,  # meters; None = native GRIB resolution
        # Crop to this bounding box before writing; omit `extents` to write full CONUS.
        extents=BoundingBox(left_lon=-122.6, right_lon=-121.9, top_lat=47.8, bottom_lat=47.3),
    )

    result = kit.download_to_dss(
        request,
        "download",
        "mrms_qpe.dss",
        grid_options=options,
        dss_version=6,
        # `force` defaults to False here and that's fine to leave as-is: MRMS
        # GRIB2 filenames encode only the product and timestamp, and the
        # downloaded grid always covers the full CONUS regardless of `extents`
        # — cropping happens later, when writing to DSS. (Contrast with HRRR,
        # where `force=True` is recommended; see the HRRR section below.)
    )

    print(f"downloaded={len(result.download.succeeded)} "
          f"written={len(result.dss.written)} "
          f"skipped={len(result.dss.skipped)} "
          f"failed={len(result.dss.failed)}")
    for pathname in result.dss.all_pathnames:
        print(pathname)
```

`QPERequest.until_now(start=...)` is a convenience constructor for archive
pulls that should run up through the current completed UTC hour.

### QPF (WPC)

Important `QPFRequest` parameters:

- `interval` — forecast accumulation window in hours: `6, 24, 48, 120`.
- `cycle_hour` — UTC forecast cycle hour, `0..23`.

```python
from pathlib import Path
from qpkit import QPFRequest, QPKit

with QPKit() as kit:
    request = QPFRequest(interval=6, cycle_hour=0)
    result = kit.download_qpf(request, Path("download"))

    print(f"downloaded={len(result.succeeded)} "
          f"skipped={len(result.skipped)} "
          f"failed={len(result.failed)}")
```

Download and write the grids to a DSS file in one step with `download_to_dss`
and `QPFGridOptions`. Each WPC QPF file is a discrete accumulation bucket; the
E part is the forecast cycle time plus the forecast hour and the D part is the
E part minus the accumulation interval:

```python
from pydsstools.core.crs import shg
from qpkit import BoundingBox, QPFRequest, QPKit
from qpkit.models import QPFGridOptions

with QPKit() as kit:
    request = QPFRequest(interval=6, cycle_hour=0)
    options = QPFGridOptions(
        part_a="WPC",
        part_b="CONUS",  # name the cropped region, not CONUS, when `extents` is set
        part_c="PRECIP",
        crs=shg(),  # destination CRS for reprojection; this is also the QPFGridOptions default
        cell_size=2500,  # meters; None = native GRIB resolution
        # Crop to this bounding box before writing; omit `extents` to write full CONUS.
        # extents=BoundingBox(left_lon=-98.0, right_lon=-94.0, top_lat=33.0, bottom_lat=29.0),
    )

    result = kit.download_to_dss(
        request,
        "download",
        "wpc_qpf.dss",
        grid_options=options,
        dss_version=6,
    )

    print(f"downloaded={len(result.download.succeeded)} "
          f"written={len(result.dss.written)} "
          f"skipped={len(result.dss.skipped)} "
          f"failed={len(result.dss.failed)}")
    for pathname in result.dss.all_pathnames:
        print(pathname)
```

### HRRR

Important `HRRRRequest` parameters:

- `precip_type` — `"APCP"` (accumulated precipitation) or `"PRATE"`
  (precipitation rate).
- `cycle_hour` — UTC hour of the model run, `0..23`.
- `cycle_date` — `date` of the run, or `None` for today (UTC).
- `forecast_hours` — tuple of lead-time hours from the cycle, each `0..48`.
- `bbox` — a `BoundingBox` cropping the NOMADS GRIB filter request. It isn't
  optional (there's no `None` form), but its default already spans the full
  CONUS extent (`left_lon=-125, right_lon=-65, top_lat=50, bottom_lat=24`), so
  omitting it downloads and writes the full CONUS grid.

Download HRRR GRIB2 files:

```python
from datetime import date
from pathlib import Path
from qpkit import BoundingBox, HRRRRequest, QPKit

download_folder = Path("download")  # GRIB2 files are saved here, named by cycle/forecast hour

with QPKit() as kit:
    request = HRRRRequest(
        precip_type="APCP",
        cycle_date=date(2026, 5, 8),
        cycle_hour=5,
        forecast_hours=(6,),
        bbox=BoundingBox(left_lon=-122.6, right_lon=-121.9, top_lat=47.8, bottom_lat=47.3),
    )
    result = kit.download_hrrr(
        request,
        download_folder,
        # The on-disk filename only encodes date/cycle/hour/precip_type, not the
        # bbox, so re-running with a different `bbox` (or while NOMADS is still
        # serving an in-progress cycle) would otherwise skip a stale local file.
        # force=True is recommended whenever the requested extent may vary
        # between downloads.
        force=True,
    )

    print(f"downloaded={len(result.succeeded)} "
          f"skipped={len(result.skipped)} "
          f"failed={len(result.failed)}")
```

Download and write the grids to DSS with `download_to_dss` and
`HRRRGridOptions`. `HRRRGridOptions.for_apcp()` / `.for_prate()` fill in the
right destination units, data type, and unit conversion for each precip type;
the `interval` argument to `download_to_dss` (HRRR `APCP` only) computes
per-interval accumulations from a single end forecast hour instead of storing
the raw running total:

```python
from datetime import date
from pathlib import Path
from pydsstools.core.crs import shg
from qpkit import BoundingBox, HRRRRequest, QPKit
from qpkit.models import HRRRGridOptions

download_folder = Path("download")  # GRIB2 files are saved here before being written to DSS
dss_file = download_folder / "precip.dss"

with QPKit() as kit:
    request = HRRRRequest(
        precip_type="APCP",
        cycle_date=date(2026, 5, 8),
        cycle_hour=5,
        forecast_hours=(6,),  # end hour; download_to_dss derives the intermediates
        bbox=BoundingBox(left_lon=-122.6, right_lon=-121.9, top_lat=47.8, bottom_lat=47.3),
    )
    options = HRRRGridOptions.for_apcp(
        part_a="HRRR",
        part_b="SEATTLE",  # name the cropped region the bbox covers, not CONUS
        part_c="PRECIP",
        part_f="HRR-APCP-1H",
        crs=shg(),  # destination CRS for reprojection; this is also the HRRRGridOptions default
        cell_size=3000,
    )

    result = kit.download_to_dss(
        request,
        download_folder,
        dss_file,
        grid_options=options,
        dss_version=6,
        # True auto-detects the latest available NOMADS cycle, overriding
        # request.cycle_date and request.cycle_hour with that cycle's values.
        cycle_hour_latest=False,
        interval=1,
        # The on-disk filename only encodes date/cycle/hour/precip_type, not the
        # bbox, so re-running with a different `bbox` (or while NOMADS is still
        # serving an in-progress cycle) would otherwise skip a stale local file.
        # force=True is recommended whenever the requested extent may vary
        # between downloads.
        force=True,
    )

    print(f"downloaded={len(result.download.succeeded)} "
          f"written={len(result.dss.written)} "
          f"skipped={len(result.dss.skipped)} "
          f"failed={len(result.dss.failed)}")
    for pathname in result.dss.all_pathnames:
        print(pathname)
```

`download_to_dss` returns a `DownloadDSSResult`, combining the download phase
(`result.download`, a `DownloadResult`) with the DSS write phase
(`result.dss`, a `DSSWriteResult` exposing `written`, `skipped`, `failed`,
`source_files`, and `all_pathnames`).

For two-stage workflows — writing GRIBs that came from elsewhere, or
re-writing an existing set of files under different DSS conventions — use the
writer services directly: `kit.hrrr_dss.write(...)` / `kit.qpe_dss.write(...)`.

## Validation and serialization

Every request (`QPERequest`, `QPFRequest`, `HRRRRequest`, `BoundingBox`) and
every result (`DownloadResult`, `DownloadDSSResult`, `DSSWriteResult`, ...) is
an immutable [pydantic](https://docs.pydantic.dev) v2 model. Two consequences
are worth knowing:

- **Requests validate at construction time.** Invalid field values or
  combinations raise `pydantic.ValidationError` immediately, before any
  network call is made — so mistakes surface at the point you build the
  request, not three steps later inside a download:

  ```python
  from qpkit import HRRRRequest

  # forecast hours must be in 0..48 — this raises pydantic.ValidationError
  # ("forecast hour 49 out of range 0..48") instead of failing during download.
  HRRRRequest(cycle_hour=0, forecast_hours=(0, 6, 49))
  ```

- **Results are easy to log or serialize.** `model_dump()` and
  `model_dump_json()` are available on every result type, including
  `DownloadResult`, `DownloadDSSResult`, and `DSSWriteResult`.
