Metadata-Version: 2.4
Name: pygndc
Version: 1.0.1
Summary: Geographic Neural Data Cube - Read and analyze .gndc compressed geospatial time-series data
Author-email: Jianbo Qi <jianboqi@126.com>
License: MIT
Project-URL: Homepage, https://geondc.org
Project-URL: Repository, https://github.com/jianboqi/pygndc
Project-URL: Issues, https://github.com/jianboqi/pygndc/issues
Keywords: geospatial,satellite,remote-sensing,neural-networks,compression
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: rasterio>=1.3.0
Requires-Dist: numpy>=1.21.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: zstandard>=0.21.0
Requires-Dist: matplotlib>=3.5.0
Requires-Dist: pillow>=9.0.0
Provides-Extra: xarray
Requires-Dist: xarray>=2022.0; extra == "xarray"
Provides-Extra: geo
Requires-Dist: geopandas>=0.12.0; extra == "geo"
Requires-Dist: shapely>=2.0.0; extra == "geo"
Provides-Extra: all
Requires-Dist: xarray>=2022.0; extra == "all"
Requires-Dist: geopandas>=0.12.0; extra == "all"
Requires-Dist: shapely>=2.0.0; extra == "all"
Requires-Dist: scipy>=1.9.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: twine>=4.0.0; extra == "dev"
Dynamic: license-file


# pygndc

**Geographic Neural Data Cube** - A Python SDK for reading and analyzing `.gndc` compressed geospatial time-series data.

<img width="286" height="320" alt="GeoNDC" src="https://github.com/user-attachments/assets/6c526878-93bc-4b70-bc99-cbb99193931a" />

## What is GeoNDC?

GeoNDC is a **continuous-time, AI-ready representation** of Earth observation archives. Unlike traditional Analysis-Ready Data (cloud-corrected raster files) or geospatial foundation model embeddings (abstract feature vectors), GeoNDC preserves the original physical observables — surface reflectance, vegetation indices, biophysical variables — while enabling millisecond-level random-access queries at any *(x, y, t)* coordinate.

Each archive (MODIS, Sentinel-2, Landsat, HiGLASS, …) is encoded into a single self-contained `.gndc` file (typically 0.5–2 GB) that runs on a laptop, a server, or directly in a browser via WebGPU. Data providers train the model once and publish the file; users download it and run inference locally — the compressed form *is* the analysis-ready form. No hosted runtime, no API quota, no vendor lock-in.

Key capabilities:

- **Continuous-time reconstruction** — query data at any moment, not just original observation times
- **Millisecond random access** — point time series in ~7 ms, full-frame reconstruction in ~2 s on a consumer GPU
- **Analytic gradients** — compute spatial/temporal derivatives directly from the neural network
- **Compact storage** — typically ~100:1 versus Int16 raster baselines, up to ~400:1 versus raw float archives
- **Implicit gap-filling** — cloud-occluded surfaces are reconstructed from the learned spatiotemporal field
- **Multi-sensor support** — Sentinel-2, Landsat, MODIS, HiGLASS, and more

## Online Viewer & Data

- **Web Viewer**: Browse `.gndc` files directly in the browser via WebGPU at [geondc.org/viewer](https://www.geondc.org/viewer/) — no installation required, GPU-accelerated, runs entirely client-side.
- **Sample Data**: Download `.gndc` datasets from [Hugging Face](https://huggingface.co/datasets/geondc/geondc-data).

## Installation

```bash
pip install pygndc
```

Or from source:

```bash
git clone https://github.com/jianboqi/pygndc.git
cd pygndc
pip install .
```

### GPU support (recommended)

`pip install pygndc` installs CPU-only PyTorch by default. For GPU acceleration, install CUDA-enabled PyTorch **first**:

```bash
# Example for CUDA 12.1 (check https://pytorch.org for your CUDA version)
pip install torch --index-url https://download.pytorch.org/whl/cu121

# Then install pygndc
pip install pygndc
```

### tiny-cuda-nn (fastest)

For best performance, install [tiny-cuda-nn](https://github.com/NVlabs/tiny-cuda-nn). This provides the optimized TCNN backend with significantly faster inference and lower memory usage.

```bash
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
```

> **Note**: tiny-cuda-nn requires an NVIDIA GPU with CUDA support and a C++ compiler. If you cannot install it, pygndc will automatically fall back to pure PyTorch (`torch_gpu` or `torch_cpu`).

### Optional dependencies

```bash
pip install pygndc[all]       # xarray + geopandas + scipy
pip install pygndc[xarray]    # xarray support
pip install pygndc[geo]       # geopandas + shapely
```

### Inference modes

pygndc uses a single `mode` parameter to control both the backend and device:

| Mode | Backend | Device | Description |
|------|---------|--------|-------------|
| `auto` | auto-detect | auto-detect | **Default.** Best available mode |
| `tcnn_cuda` | tiny-cuda-nn | GPU | Fastest. Requires tinycudann + CUDA |
| `torch_gpu` | PyTorch | GPU | No extra dependencies, requires CUDA |
| `torch_cpu` | PyTorch | CPU | Works everywhere, slowest |

## Quick Start

```python
import pygndc

# Open a .gndc file (auto-detects best mode)
with pygndc.open("samples/S2_sample_2022_to_2025.gndc") as ds:
    print(ds)                                          # Shows mode, shape, CRS, etc.

    # --- Read OBSERVED data: read(time, <spatial selector>, bands) ---
    frame  = ds.read(time=0)                               # full frame (H, W, C)
    frame  = ds.read(time="2024-06-15")                    # by stored date (exact match)
    sub    = ds.read(time=0, window=(col_off, row_off, w, h))      # pixel window
    region = ds.read(time=0, bbox=(116.3, 39.8, 116.5, 40.0))     # geographic bbox
    px     = ds.read(time=0, rowcol=(140, 140))            # pixel point -> (C,)
    px     = ds.read(time=0, latlng=(116.825, 40.486))     # geographic point -> (C,)
    rgb    = ds.read(time=0, bands=[0, 1, 2])              # select bands

    # Windowed reads build coords for just the window — memory scales with the
    # window, not the full frame, so huge images don't OOM.

    # --- SYNTHESIZE an unobserved time: interpolate(...) ---
    # read() raises on an unobserved date; interpolate() opts into synthesis.
    synth  = ds.interpolate("2024-06-15 12:00:00")         # continuous time interpolation
    synth  = ds.interpolate(0.5, bbox=(116.3, 39.8, 116.5, 40.0))  # 0.5 = normalized [0,1]

    # --- Point time series: series(point selector) -> (T, C) ---
    ts     = ds.series(latlng=(116.825, 40.486))           # (T, C) at a geo point
    ts     = ds.series(rowcol=(140, 140))                  # (T, C) at a pixel

    # Analysis
    ndvi = ds.ndvi(t=0, red_band=2, nir_band=3)            # (H, W)
    dx, dy = ds.gradient(t=0, mode="spatial")              # analytic gradients

    # Export
    arr = ds.to_numpy(t=0)                                 # (H, W, C) numpy array
    ds.to_tif("frame_0.tif", t=0)                         # single frame to GeoTIFF
    ds.to_tifs("output_dir/")                              # all frames
    xds = ds.to_xarray()                                   # xarray Dataset

# Specify mode explicitly
with pygndc.open("samples/S2_sample_2022_to_2025.gndc", mode="torch_cpu") as ds:
    frame = ds.read(time=0)
```

> **API note (v0.7):** data access was unified into three verbs — `read()`
> (observed time), `interpolate()` (synthesized time), `series()` (point time
> series) — each taking exactly one spatial selector (`window=` / `bbox=` /
> `rowcol=` / `latlng=`). This replaces the previous `read(t=...)`,
> `read_at_time()`, `read_region()`, `sample()` and `pixel_series()` methods.

## CLI Commands

```bash
# Show model info
pygndc info model.gndc
pygndc info model.gndc --json

# Decompress to GeoTIFF files
pygndc decompress -i model.gndc -o ./output
pygndc decompress -i model.gndc -o ./output --timestamp 2024-06-15
pygndc decompress -i model.gndc -o ./output --start 2024-01-01 --end 2024-12-31 --interval 5

# Compute NDVI
pygndc ndvi -i model.gndc -o ndvi.tif --red-band 0 --nir-band 1

# Compute spatial gradient
pygndc derivative -i model.gndc -o gradient.tif --grad-mode mag

# Point query
pygndc sample -i model.gndc --lon 116.5 --lat 39.9

# Export pixel time series to CSV
pygndc timeseries -i model.gndc --row 100 --col 200 -o series.csv

# Launch interactive viewer
pygndc viewer -i model.gndc

# All commands support --mode (auto, tcnn_cuda, torch_gpu, torch_cpu)
pygndc decompress -i model.gndc -o ./output --mode torch_cpu
```

## Interactive Viewer

A GUI tool for visualizing `.gndc` files with multiple display modes, time navigation, and pixel inspection.

```python
from pygndc import start_viewer
start_viewer()
```

Features:
- RGB, NDVI, temporal gradient, and spatial gradient display modes
- Time slider with date navigation
- Click-to-inspect pixel time series
- Region selection and zoom
- Export current frame or entire time series

## File Format

A `.gndc` file is a zstandard-compressed archive containing:

| Component | Description |
|-----------|-------------|
| Header | JSON metadata (dimensions, timestamps, CRS, band info) |
| Model Weights | Compressed neural network parameters |
| Mask Data | Optional validity mask (static or dynamic) |
| Residuals | Optional correction data for higher accuracy |

## API Reference

See [API_Reference.md](API_Reference.md) for the full API documentation.

## License

MIT License

## Citation

```bibtex
@misc{qi2026geondcqueryableneuraldata,
  title={GeoNDC: A Queryable Neural Data Cube for Planetary-Scale Earth Observation},
  author={Jianbo Qi and Mengyao Li and Baogui Jiang and Yidan Chen and Qiao Wang},
  year={2026},
  eprint={2603.25037},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2603.25037},
}
```

## Contact

- Author: Jianbo Qi
- Email: jianboqi@126.com
- Issues: [GitHub Issues](https://github.com/jianboqi/pygndc/issues)
