Metadata-Version: 2.4
Name: sverdrup
Version: 0.2.0
Summary: Regional sea-surface-height-anomaly reconstruction with first-class predictive uncertainty
Project-URL: Homepage, https://github.com/killett/sverdrup
Project-URL: Repository, https://github.com/killett/sverdrup
Project-URL: Issues, https://github.com/killett/sverdrup/issues
Author-email: Emmy Killett <emmykillett@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: gaussian process,oceanography,optimal interpolation,sea surface height,uncertainty quantification
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: numpy>=1.26
Requires-Dist: pyproj>=3.6
Requires-Dist: scipy>=1.11
Provides-Extra: all
Requires-Dist: dask[distributed]>=2024.1; extra == 'all'
Requires-Dist: fsspec>=2024.1; extra == 'all'
Requires-Dist: netcdf4>=1.6; extra == 'all'
Requires-Dist: requests>=2.31; extra == 'all'
Requires-Dist: tenacity>=8.2; extra == 'all'
Requires-Dist: xarray>=2024.1; extra == 'all'
Provides-Extra: dask
Requires-Dist: dask[distributed]>=2024.1; extra == 'dask'
Provides-Extra: io
Requires-Dist: fsspec>=2024.1; extra == 'io'
Requires-Dist: netcdf4>=1.6; extra == 'io'
Requires-Dist: requests>=2.31; extra == 'io'
Requires-Dist: tenacity>=8.2; extra == 'io'
Requires-Dist: xarray>=2024.1; extra == 'io'
Description-Content-Type: text/markdown

# sverdrup

**sverdrup** reconstructs regional sea-surface-height anomaly (SSHA) fields from sparse
satellite altimetry, with *first-class, rigorous per-gridpoint uncertainty* — every output is a
predictive distribution (mean + exact marginal variance + coherent whole-field samples + a typed
provenance chain), not just a point estimate. It ships three interchangeable methods behind one
method-agnostic spine — a dense space-time Gaussian-process / **optimal interpolation** (`oi`), a
sparse-precision **Matérn GMRF** (`gmrf`, incl. latitude-varying correlation length), and a
**trivial** inverse-distance baseline (`trivial`) — and blends overlapping tiles into one
seam-free regional product with coherent cross-tile uncertainty. Reconstructions are scored two
honest ways: **OSSE** (against gridded truth) and **OSE** (against withheld along-track data).

## Table of contents

- [Quickstart](#quickstart)
- [Installation](#installation)
- [Concepts in 60 seconds](#concepts-in-60-seconds)
- [Running a reconstruction](#running-a-reconstruction)
  - [CLI (config file)](#cli-config-file)
  - [Config keys](#config-keys)
  - [Programmatic API](#programmatic-api)
  - [Methods and their parameters](#methods-and-their-parameters)
- [Cheatsheet](#cheatsheet)
- [Output](#output)
- [Troubleshooting](#troubleshooting)
- [Links and license](#links-and-license)

## Quickstart

```bash
pip install 'sverdrup[all]'
```

Reconstruct an OSSE tile on the committed tiny fixtures and print the scores:

```python
from sverdrup.adapters.odc.fixtures import FixtureSource
from sverdrup.application.pipeline import PipelineInputs, run_pipeline

src = FixtureSource(
    "tests/fixtures/natl60_tiny.nc", ref_path="tests/fixtures/natl60_ref_tiny.nc"
)
product, scores = run_pipeline(PipelineInputs(
    mode="OSSE", method_name="oi", source=src, out_url="file:///tmp/osse.zarr",
    lon_range=(-64, -56), lat_range=(34, 42), time_range=(0, 5), output_times=[2.0],
    params={"length_scale": 300.0, "time_scale": 10.0, "variance": 0.05},
))
print(scores)  # RMSE vs truth, calibration (reduced chi^2, coverage), ground-track power
```

Or run the same thing from the shipped config:

```bash
python -m sverdrup tests/integration/config_osse.json
```

> The `tests/...` fixture paths are repo-relative — clone the repo to run these examples
> as-is, or point `obs_path` / `ref_path` (and the `FixtureSource` arguments) at your own
> NetCDF observations.

## Installation

Requires **Python ≥ 3.12**.

```bash
pip install sverdrup            # core library API only (lightweight)
pip install 'sverdrup[all]'     # everything needed to RUN the pipeline (recommended)
```

Extras (install only what you need):

| Extra     | Pulls in                                       | Needed for |
|-----------|------------------------------------------------|------------|
| `[dask]`  | `dask[distributed]`                            | the parallel executor (any pipeline run) |
| `[io]`    | `xarray`, `fsspec`, `netcdf4`, `requests`, …   | reading NetCDF obs + writing the zarr product |
| `[all]`   | `[dask]` + `[io]`                              | running reconstructions end-to-end |

> The bare `import sverdrup` surface (core grid/types/distributions) works with no extras. The
> pipeline (`run_pipeline`, `python -m sverdrup`) eagerly imports the dask executor and the I/O
> stack, so it needs `[all]`.

**conda-forge:** packaging is in progress (a `staged-recipes` PR is open). Once merged,
`conda install -c conda-forge sverdrup` will track every PyPI release automatically.

## Concepts in 60 seconds

- **Mode — OSSE vs OSE.** `OSSE` scores the reconstructed grid against gridded "truth" (accuracy
  + calibration). `OSE` withholds a real mission (CryoSat-2) from training and scores the
  reconstruction against that withheld along-track data — no truth leak.
- **Method.** `oi` (exact dense GP, full space-time kernel), `gmrf` (sparse-precision Matérn
  SPDE, fast + nonstationary-capable), `trivial` (inverse-distance baseline / degradation path).
- **The `Product`.** Each output time carries a *Persisted* predictive distribution: mean, **exact**
  marginal variance, coherent whole-field samples, off-grid eval-point predictions, and a typed
  **provenance** chain that records every uncertainty transform and any known bias (e.g. a
  conservative halo residual, or `DEGRADED_COHERENCE` on the trivial path).
- **Tiling and blend.** A region is split into overlapping tiles, each solved independently, then
  crossfaded into one seam-free product. `oi`/`gmrf` keep cross-tile uncertainty **coherent**; the
  `trivial` path is honestly **degraded** and flags the coherence loss in provenance.

## Running a reconstruction

### CLI (config file)

```bash
python -m sverdrup <config.json>
```

The shipped example, `tests/integration/config_osse.json`:

```json
{
  "mode": "OSSE",
  "method": "oi",
  "obs_path": "tests/fixtures/natl60_tiny.nc",
  "ref_path": "tests/fixtures/natl60_ref_tiny.nc",
  "out_url": "file:///tmp/sverdrup_osse.zarr",
  "lon_range": [-64, -56],
  "lat_range": [34, 42],
  "time_range": [0, 5],
  "output_times": [2.0],
  "params": {"length_scale": 300.0, "time_scale": 10.0, "variance": 0.05},
  "grid_resolution_deg": 1.0,
  "executor": {"n_processes": 2, "threads_per_process": 1},
  "rank": 20
}
```

### Config keys

| Key                  | Type            | Meaning |
|----------------------|-----------------|---------|
| `mode`               | `"OSSE"`/`"OSE"` | scoring mode (gridded truth vs withheld along-track) |
| `method`             | `"oi"`/`"gmrf"`/`"trivial"` | reconstruction method |
| `obs_path`           | path            | observation NetCDF |
| `ref_path`           | path (optional) | gridded-truth NetCDF (OSSE) |
| `out_url`            | fsspec URL      | where the zarr product is written (`file://`, `s3://`, …) |
| `lon_range`/`lat_range` | `[min, max]` | region bounds in degrees |
| `time_range`         | `[min, max]`    | training window in days |
| `output_times`       | `[float, …]`    | output time(s) in days |
| `params`             | object          | method parameters (see table below) |
| `grid_resolution_deg`| float (def `1.0`) | target grid spacing |
| `executor`           | object          | `{n_processes, threads_per_process}` for the dask LocalCluster |
| `rank`               | int (def `20`)  | low-rank factor rank for the persisted product |

### Programmatic API

Single tile — `run_pipeline(PipelineInputs(...)) -> (product, scores)` (see [Quickstart](#quickstart)).

Regional, multi-tile coherent blend — `run_tiled_pipeline(inputs, partition)`:

```python
from sverdrup.application.pipeline import PipelineInputs, run_tiled_pipeline
from sverdrup.application.tiling import LonLatPartition, ScaleAwareHalo
from sverdrup.adapters.executor_dask import ExecutorConfig
from sverdrup.adapters.odc.fixtures import FixtureSource
from sverdrup.core.parameters import ConstantProvider

inp = PipelineInputs(
    mode="OSSE", method_name="gmrf",
    source=FixtureSource("tests/fixtures/natl60_tiny.nc",
                         ref_path="tests/fixtures/natl60_ref_tiny.nc"),
    out_url="file:///tmp/blend.zarr",
    lon_range=(-64, -56), lat_range=(34, 42), grid_resolution_deg=1.0,
    time_range=(0, 5), output_times=[2.0],
    params={"range": 300.0, "variance": 0.05, "temporal_taper_scale": 10.0},
    executor=ExecutorConfig(n_processes=2, threads_per_process=1), rank=20,
)
partition = LonLatPartition(
    n_lon=3, n_lat=1, halo=ScaleAwareHalo(k=1.0),
    correlation_length=ConstantProvider({"correlation_length": 300.0}),
    stencil_radius_km=10.0,
)
blends, scores = run_tiled_pipeline(inp, partition)   # one BlendedDistribution per output time
```

### Methods and their parameters

| `method`   | `params`                                                  | Notes |
|------------|-----------------------------------------------------------|-------|
| `oi`       | `length_scale` (km), `time_scale` (days), `variance`      | exact dense space-time GP / OI |
| `gmrf`     | `range` (km), `variance`, `temporal_taper_scale` (days)   | sparse Matérn SPDE; `range` may be a latitude-varying field (nonstationary κ) |
| `trivial`  | `{}` (none)                                               | inverse-distance baseline; degradation path |

## Cheatsheet

```bash
# install (everything needed to run)
pip install 'sverdrup[all]'

# run a reconstruction from a config file
python -m sverdrup path/to/config.json

# minimal OSSE config: copy tests/integration/config_osse.json and edit
#   mode/method/paths/ranges/output_times/params

# methods + params
#   oi      -> {"length_scale": 300.0, "time_scale": 10.0, "variance": 0.05}
#   gmrf    -> {"range": 300.0, "variance": 0.05, "temporal_taper_scale": 10.0}
#   trivial -> {}

# OSE (withheld along-track) instead of OSSE: set "mode": "OSE" and drop ref_path
```

## Output

`run_pipeline` returns `(product, scores)`; `run_tiled_pipeline` returns
`(blends, scores)`. The product is written to `out_url` as zarr (via fsspec, so local
`file://`, `s3://`, `gcs://`, … all work). Each per-time entry carries the mean, exact marginal
variance, coherent samples, off-grid eval-point predictions, and the typed uncertainty
**provenance** (every transform + any `KnownBias`). `scores` is the merged evaluator dictionary:
RMSE vs truth, calibration (reduced χ², 1σ coverage) for OSSE; withheld-track RMSE for OSE; plus
ground-track power.

## Validation — reproduces the 2021a SSH-mapping OSE BASELINE

Sverdrup's OI engine reproduces the published **2021a SSH Mapping Data Challenge**
BASELINE leaderboard row, scored by the challenge's *own* evaluation code (vendored
as a submodule and driven through `sverdrup.validation`):

| Method | µ(RMSE) | σ(RMSE) | λx (km) |
|--------|---------|---------|---------|
| **sverdrup OI** | **0.853** | **0.090** | **140.9** |
| BASELINE (published) | 0.85 | 0.09 | 140 |
| DUACS (published) | 0.88 | 0.07 | 152 |

**Verdict: PASS** (µ tolerance ±0.03, never loosened). The challenge's eval is
independently trusted — it reproduces the published DUACS, MIOST, and BFN rows to
within tolerance — and our own parallel skill score agrees with it on our map
(Δ 0.005). The reproduction required a faithful Gaussian, anisotropic,
degree-space kernel (`GaussianSpaceTimeDegrees`) and the SLA→SSH MDT reference
frame; details, the decomposed read, and the data-source notes are in the
[`docs/validation/`](docs/validation/) records below.

> **`sverdrup.validation` is a from-source tool, not part of the PyPI/conda
> package.** It needs the vendored challenge submodule (`git submodule update
> --init`), the downloaded challenge data, and extra deps (`httpx`, `stamina`,
> `paramiko`, `pyinterp`) that ship only in the pixi dev environment — so it is
> not exercised by a bare `pip install sverdrup`. Run it from a clone.

## Troubleshooting

| Symptom | Fix |
|---------|-----|
| `ModuleNotFoundError: No module named 'dask'` (or `xarray`, `fsspec`, `netcdf4`) | Install the run extras: `pip install 'sverdrup[all]'`. The core install is deliberately light. |
| Oracle / ODC-data tests are **skipped** | Expected offline. They are opt-in: set `SVERDRUP_ODC_DATA` to a cached NATL60 window — see [`docs/oracle-runbook.md`](docs/oracle-runbook.md). |
| Want output somewhere other than local disk | `out_url` is any fsspec URL — `s3://bucket/key.zarr`, `gcs://…`, etc. (install the matching fsspec backend). |
| Reconstruction is slow / over-subscribes CPUs | Tune `executor` (`n_processes`, `threads_per_process`). One BLAS thread per worker is usually fastest; set thread env vars *before* numpy/BLAS loads. |
| macOS install fails to solve a Linux-only dependency | A few conda deps are linux-64-only; this affects the pixi dev environment, not the PyPI install. Use `pip install 'sverdrup[all]'`. |

## Links and license

- [`docs/validation/RESULT.md`](docs/validation/RESULT.md) — the 2021a BASELINE reproduction result (table + decomposed read); [`docs/validation/parameter_audit_trail.md`](docs/validation/parameter_audit_trail.md) — full parameter/eval/MDT audit trail
- [`docs/oracle-runbook.md`](docs/oracle-runbook.md) — opt-in correctness oracle (reproduce the ODC OI leaderboard number)
- [`docs/`](docs/) — architecture design + implementation plans; [`conda-recipe/`](conda-recipe/) — conda-forge packaging
- `PROGRESS.md` — running project notebook (decisions, gotchas, deviations)

Licensed under **Apache-2.0**.
