Metadata-Version: 2.4
Name: coglite
Version: 0.1.1
Summary: Read windows, pixels and overviews from Cloud-Optimized GeoTIFFs over HTTP(S) or local files with numpy only - no GDAL, no rasterio.
Author-email: Yusuf Eminoğlu <yusufeminoglu@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/YusufEminoglu/coglite_sdk
Project-URL: Repository, https://github.com/YusufEminoglu/coglite_sdk
Project-URL: Issues, https://github.com/YusufEminoglu/coglite_sdk/issues
Project-URL: Changelog, https://github.com/YusufEminoglu/coglite_sdk/blob/main/CHANGELOG.md
Keywords: cog,cloud-optimized-geotiff,geotiff,tiff,bigtiff,raster,gis,range-request,serverless,pyodide,remote-sensing,web-mercator
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20.0
Provides-Extra: imagecodecs
Requires-Dist: imagecodecs>=2021.11.20; extra == "imagecodecs"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Provides-Extra: crossval
Requires-Dist: rasterio>=1.3.0; extra == "crossval"
Dynamic: license-file

# coglite

[![CI](https://github.com/YusufEminoglu/coglite_sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/YusufEminoglu/coglite_sdk/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/coglite.svg)](https://pypi.org/project/coglite/)
[![Python](https://img.shields.io/pypi/pyversions/coglite.svg)](https://pypi.org/project/coglite/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

**Read windows, pixels and overviews from Cloud-Optimized GeoTIFFs over HTTP(S) or
local files, with numpy as the only dependency.** No GDAL, no rasterio, no PROJ,
no compiled extensions.

coglite is meant for places where installing GDAL is painful or impossible:
AWS Lambda / Cloud Functions, slim Docker images, Pyodide / JupyterLite, CI jobs
and small tools that only need to pull a few tiles out of a large raster.

```python
import coglite

url = "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/36/Q/WD/2020/7/S2A_36QWD_20200701_0_L2A/B04.tif"
with coglite.open(url) as cog:
    print(cog.profile)            # width, height, dtype, compression, block size, overviews ...
    print(cog.crs_epsg, cog.bounds)
    arr = cog.read(window=(4096, 4096, 1024, 1024))      # (bands, rows, cols) numpy array
    thumb = cog.read_bbox(cog.bounds, max_size=512)      # picks the best overview
    print(thumb.level, thumb.data.shape, thumb.transform)
    print(cog.point(550000, 1850000))                    # values at a map coordinate
    print(cog.reader.stats)                              # requests / bytes actually fetched
```

Opening that 10980 x 10980 Sentinel-2 band costs one 16 KiB request; the
1024 x 1024 window above is one more request of 1.3 MB.

## Installation

```bash
pip install coglite                 # numpy only
pip install "coglite[imagecodecs]"  # + JPEG, WebP, ZSTD and LERC decoding
```

Python 3.9 or newer.

## What is supported

| Area | Supported |
| --- | --- |
| Containers | classic TIFF and BigTIFF, little- and big-endian |
| Structure | IFD chains, SubIFDs, overviews (`NewSubfileType` reduced resolution), GDAL internal masks (1-bit or 8-bit, per overview), sparse (empty) blocks |
| Layout | tiled and stripped, `PlanarConfiguration` chunky (pixel interleaved) and separate (band interleaved), any number of bands |
| Sample types | unsigned and signed integers of 8, 16, 32, 64 bits; float 16/32/64; 1-bit masks |
| Built-in codecs | none, Deflate/zlib (8 and 32946), LZW (pure Python), PackBits, LZMA (standard library) |
| Optional codecs | JPEG (incl. YCbCr), WebP, ZSTD, LERC (+Deflate / +ZSTD) through `imagecodecs`; ZSTD also through the standard library on Python 3.14+ |
| Predictors | 1 (none), 2 (horizontal, integers and floats), 3 (floating point) |
| GeoTIFF | `ModelPixelScale` + `ModelTiepoint`, `ModelTransformation`, `GeoKeyDirectory` (+ double and ASCII params) to an EPSG code, `PixelIsPoint` half-pixel shift (as GDAL), `GDAL_NODATA` (incl. `nan`), `GDAL_METADATA` scale/offset and items |
| COG layout | GDAL ghost area (`GDAL_STRUCTURAL_METADATA`), block leader/trailer checks, validation of IFD and data order |
| I/O | local files (mmap, falls back to seek/read), `bytes`, HTTP(S) via `urllib` with Range requests, presigned URLs, any custom `fetch(offset, length)` |

Pixel values are verified to be identical to GDAL's (read through rasterio) for
every combination above, including overviews and masks; see
[`tests/test_gdal_crossval.py`](tests/test_gdal_crossval.py).

## API

### `coglite.open(source, **options) -> COG`

`source` may be a path, an `http(s)://` or `file://` URL, `bytes`, or a
`RangeReader`. HTTP options: `headers`, `timeout` (s, default 30), `retries`
(default 3), `max_workers` (default 4), `merge_gap` (bytes, default 64 KiB),
`cache_bytes` (default 32 MiB). `prefetch` (default 16 KiB) is the size of the
first header request.

### `COG`

| Member | Description |
| --- | --- |
| `width`, `height`, `count`, `dtype` | raster size, band count, numpy dtype name |
| `transform` | `Affine(a, b, c, d, e, f)` (rasterio order); `.apply(col, row)`, `.inverse()`, `.to_gdal()` |
| `bounds` | `(left, bottom, right, top)` in the raster CRS |
| `crs_epsg` | EPSG code from the GeoKeys (None for user-defined CRSs) |
| `nodata`, `scales`, `offsets`, `metadata`, `band_metadata`, `geokeys` | GDAL metadata |
| `profile`, `info()` | summary dicts (`info()` is JSON-serialisable) |
| `overviews`, `levels` | `Level(index, width, height, block_width, block_height, transform, decimation)`; `levels[0]` is full resolution |
| `has_mask` | an internal mask is present |
| `read(window=None, bands=None, overview=None, *, masked=False, fill_value=None, apply_scale=False)` | `(bands, rows, cols)` array. `window` is `Window(col_off, row_off, width, height)` or a 4-tuple in pixels of the chosen level; parts outside the image are filled. `bands` is 1-based (an int returns a 2-D array). `overview=0` is the first overview. `masked=True` returns a `numpy.ma.MaskedArray` (internal mask, nodata, outside pixels). `apply_scale=True` returns `value * scale + offset` as float64. |
| `read_mask(window=None, overview=None)` | uint8 validity mask (255 valid, 0 invalid) |
| `read_bbox(bbox, crs=None, max_size=1024, bands=None, *, masked=False)` | reads the pixels covering `bbox`, choosing the finest level whose window fits in `max_size` (longest side); decimates with nearest neighbour only when even the coarsest overview is too large. Returns `RasterResult(data, transform, crs_epsg, level, window)` with `.bounds`. |
| `point(x, y, crs=None, bands=None, overview=None)` | values at a map coordinate, one per band |
| `tile(z, x, y, tilesize=256, bands=None, *, masked=False, fill_value=None)` | XYZ Web Mercator tile with nearest-neighbour sampling, **only for COGs that are themselves in EPSG:3857** |
| `validate(check_blocks=True)` | `ValidationReport(errors, warnings, details)`; `.is_valid`, `.to_dict()` |
| `reader.stats` | `requests`, `bytes_fetched`, `cache_hits`, `ranges_requested`, `retries` |

### Coordinate systems: know the limits

coglite has no PROJ. `crs=` arguments accept only the raster's own EPSG code or
the EPSG:4326 / EPSG:3857 pair (converted with the closed-form spherical Mercator
formulas). Anything else raises `CRSError`; reproject your bbox first, e.g. with
`pyproj`. `tile()` performs no reprojection, so it only works on EPSG:3857 COGs
(also 900913, 3785, 102100, 102113), and it never resamples other than nearest
neighbour. Rotated or sheared rasters can be read with `read()`, but
`read_bbox()` and `tile()` need a north-up transform.

## Range requests, coalescing, caching

All block reads of one `read()` call are gathered and handed to the reader in
one batch. The reader

1. returns blocks already in its LRU cache (32 MiB by default),
2. merges ranges whose gap is at most `merge_gap` bytes (64 KiB by default for
   HTTP) into one request of at most `max_request` bytes (16 MiB),
3. issues the merged requests concurrently with up to `max_workers` threads
   (4 by default, a polite value for public buckets).

COGs store the tiles of a row next to each other, so a typical window costs one
request per few tile rows, or a single request. Measured against the local test
server (512 x 512 COG, 32 x 32 tiles):

| Read | coalescing off (`merge_gap=-1`) | default |
| --- | --- | --- |
| 3 x 3 tile window | 9 requests | 1 request |
| full image (256 tiles) | 256 requests | 1 request |
| repeated window | 0 requests (cache) | 0 requests (cache) |

The header is probed with one request of `prefetch` bytes; if the IFDs are
bigger, the buffer grows (at least doubling) instead of issuing many tiny
requests. GDAL-written COGs keep all IFDs at the start of the file, so opening
costs one request, and blocks that already arrived with the header buffer are
never fetched again (a COG smaller than `prefetch` is read with one request).

HTTP behaviour: only `GET` with a `Range` header is used (never `HEAD`), so URLs
presigned for `GET` work unchanged. Timeouts, connection errors, HTTP 429 and
5xx are retried with exponential backoff; other HTTP errors raise
`RangeRequestError` at once. A server that ignores `Range` and answers `200` is
handled by keeping the body in memory.

### Plugging in other storage

```python
# S3 through boto3 (or anything that can read a byte range)
import boto3, coglite
s3 = boto3.client("s3")

def fetch(offset, length):
    r = s3.get_object(Bucket="my-bucket", Key="dem.tif",
                      Range=f"bytes={offset}-{offset + length - 1}")
    return r["Body"].read()

cog = coglite.open(coglite.CallableReader(fetch, name="s3://my-bucket/dem.tif"))

# or presign and use plain HTTPS
url = s3.generate_presigned_url("get_object", Params={"Bucket": "my-bucket", "Key": "dem.tif"})
cog = coglite.open(url)

# or register a URL scheme once
coglite.register_scheme("s3", lambda url, **kw: coglite.HTTPReader(presign(url), **kw))
cog = coglite.open("s3://my-bucket/dem.tif")
```

Subclass `coglite.RangeReader` and implement `_fetch(offset, length)` (and
optionally `size`) for anything else; caching, coalescing and concurrency come
for free.

### Pyodide / WebAssembly

Threads are not available in Pyodide: pass `max_workers=1`. Synchronous
`urllib` needs the `pyodide-http` patch (`pyodide_http.patch_all()`), or use a
`CallableReader` around a synchronous `XMLHttpRequest`, or fetch the file
yourself and pass the `bytes`.

## Command line

```bash
coglite info https://example.com/cog.tif            # profile, bounds, overviews, I/O used
coglite info cog.tif --json
coglite read cog.tif --bbox 500000 4090000 501000 4100000 --max-size 512 -o out.png
coglite read cog.tif --bbox 28.9 41.0 29.1 41.1 --crs EPSG:4326 -o out.npy   # 3857 COGs
coglite read cog.tif --window 0 0 256 256 --overview 1 --bands 3,2,1 -o rgb.png
coglite validate https://example.com/cog.tif        # exit code 1 when not a valid COG
```

PNG output is stretched to 8 bits per band (min/max of valid pixels); masked
pixels become transparent. `--header "Authorization: Bearer ..."` adds HTTP headers.

## COG validation rules

`COG.validate()` follows GDAL's `validate_cloud_optimized_geotiff.py`:

* errors: images (or overviews) larger than 512 x 512 that are not tiled; the
  first IFD not at the start of the file; IFDs not ordered full resolution then
  decreasing overviews; IFDs located after image data; image data not stored from
  the smallest overview to full resolution; a ghost area declaring
  `KNOWN_INCOMPATIBLE_EDITION=YES`; block leaders/trailers that contradict the
  ghost area; non row-major block order when the ghost area promises it;
* warnings: no overviews on large images; tile sizes not multiples of 16 or not
  powers of two; non row-major block order; overviews stored in SubIFDs.

## Performance notes and limitations

* Deflate, LZMA and PackBits decode at native speed. LZW is pure Python
  (about 0.6 s per MiB of noisy data; much faster on smooth data). Prefer Deflate
  or ZSTD when you write COGs for coglite.
* No resampling beyond nearest neighbour and no reprojection.
* Not supported: bit depths other than 1/8/16/32/64, palette expansion
  (`ColorMap` is ignored; indices are returned), old-style JPEG (compression 6),
  JPEG-XL, CCITT fax codecs, sample-format mixtures inside one image.
* Multi-page TIFFs: only the first page and its reduced-resolution pages are used.

## Development

```bash
pip install -e ".[dev]"
pytest                                   # GDAL cross-validation runs when gdal_translate + rasterio exist
ruff check . && ruff format --check src tests && mypy src
```

`tests/data/` holds four tiny GDAL-written fixtures (78 KB in total, regenerated
with `tests/data/make_fixtures.py`). Larger cross-validation files are generated
at test time when `gdal_translate` is available (set `COGLITE_GDAL_TRANSLATE` to
its path, or `COGLITE_NO_GDAL=1` to skip them). HTTP tests use a local
standard-library server; the test suite never touches the internet.

## License

MIT, see [LICENSE](LICENSE). Citation metadata is in [CITATION.cff](CITATION.cff).
