Metadata-Version: 2.4
Name: geoembed
Version: 1.1.1
Summary: netcl visual feature encoders for aerial and satellite imagery
License: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.22
Requires-Dist: Pillow>=9.0
Requires-Dist: netcl>=0.4.1

# GeoEmbed

netcl-based visual feature encoders and continuous latent field projections for aerial and satellite imagery.

---

## Core Capabilities & Metrics

* **256d L2-Normalized Vectors**: Encodes aerial imagery into compact 256-dimensional unit vectors ($\|z\|_2 = 1.0$).
* **Distance & Similarity Metrics**: Because all vectors lie on the unit hypersphere, inner product directly computes cosine similarity in $[-1.0, 1.0]$:

$$S(z_1, z_2) = z_1 \cdot z_2$$

Euclidean distance is strictly monotonically related to cosine distance, making the embeddings directly compatible with standard vector indexing (FAISS, Qdrant, Milvus):

$$D_{\text{euc}}(z_1, z_2) = \sqrt{2(1 - z_1 \cdot z_2)}$$

* **Continuous Latent Field PCA Projection**: Maps 256d embeddings into continuous 3-channel RGB color space via SVD-based PCA with robust 2%–98% percentile contrast stretching. Semantically similar landscapes receive visually neighboring hues.
* **Topological & Semantic Structure**: Captures road networks, parcel boundaries, forested areas, water bodies, and urban patterns without manual supervision.
* **Environmental & Temporal Invariance**: Robust across multi-year temporal drift, seasonal foliage variation, sun angles, and illumination shifts.
* **Asynchronous Multi-Threaded Pipelining**: Background CPU worker threads continuously load and preprocess tiles ahead of GPU execution (`workers=4`), maintaining high inference throughput without I/O stalls.

---

## Visual Showcase

### Urban Infrastructure & Topology Emergence (Stuttgart Center, $z=20$)

<table>
  <tr>
    <th width="25%" align="center">Reference Orthophoto</th>
    <th width="25%" align="center">V3 Baseline (ResNet-18)</th>
    <th width="25%" align="center">V11 Dual-Tower</th>
    <th width="25%" align="center">V12 Refined Topology</th>
  </tr>
  <tr>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example12.png" width="100%" alt="Reference Orthophoto"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example13.png" width="100%" alt="V3 Baseline"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example14.png" width="100%" alt="V11 Dual-Tower"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example15.png" width="100%" alt="V12 Refined Topology"></td>
  </tr>
</table>

In dense urban environments, the relational self-attention heads in **V11** and **V12** delineate continuous infrastructure: arterial road networks, railway corridors, and park axes emerge in vivid contrast without any manual annotations or labels.

### Unsupervised Landscape Clustering (Rural Terrain, $z=20$)

The same embeddings group natural terrain types and agricultural parcels across cluster counts $k$:

<table>
  <tr>
    <th width="33.33%" align="center">Reference Orthophoto</th>
    <th width="33.33%" align="center">Continuous Latent Field (PCA)</th>
    <th width="33.33%" align="center">Cluster Map (k = 2)</th>
  </tr>
  <tr>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example11.png" width="100%" alt="Reference Orthophoto"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example1.png" width="100%" alt="Continuous Field"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example2.png" width="100%" alt="Cluster k=2"></td>
  </tr>
  <tr>
    <th align="center">Cluster Map (k = 3)</th>
    <th align="center">Cluster Map (k = 4)</th>
    <th align="center">Cluster Map (k = 6)</th>
  </tr>
  <tr>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example3.png" width="100%" alt="Cluster k=3"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example4.png" width="100%" alt="Cluster k=4"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example6.png" width="100%" alt="Cluster k=6"></td>
  </tr>
  <tr>
    <th align="center">Cluster Map (k = 8)</th>
    <th align="center">Cluster Map (k = 10)</th>
    <th align="center">Cluster Map (k = 20)</th>
  </tr>
  <tr>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example7.png" width="100%" alt="Cluster k=8"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example8.png" width="100%" alt="Cluster k=10"></td>
    <td><img src="https://raw.githubusercontent.com/GodOfLores/geoembed/main/example10.png" width="100%" alt="Cluster k=20"></td>
  </tr>
</table>

At low cluster counts ($k=2, 3, 4$), the latent space cleanly partitions macroscopic landscape classes (dense forest vs. open agricultural land). As $k$ increases ($k=6, 8, 10, 20$), subtle sub-structures such as access roads, parcel boundaries, and individual built structures emerge purely unsupervised.

> *Data source of sample orthophotos: © LGL Baden-Württemberg (www.lgl-bw.de), licensed under dl-de/by-2-0. See [ATTRIBUTION.md](https://github.com/GodOfLores/geoembed/blob/main/ATTRIBUTION.md) for details.*

---

## What It Expects as Input

* **Image Type**: Nadir aerial and satellite imagery (orthophotos, vertical top-down view). Oblique drone or street-level photos are not supported.
* **Geometry**: Square image tiles (automatically resized to native resolution: $128 \times 128$ for V3, $256 \times 256$ for V11/V12).
* **Format**: File paths (`str` / `Path`), raw image bytes (`bytes` / `bytearray` / SQLite BLOBs), PIL `Image`, or NumPy arrays. Inputs are automatically normalized to 3-channel RGB float32 in $[-1.0, 1.0]$.
* **Clear Preference for `z=20` Web-Mercator**:
  * **Optimal Ground Scale**: **`z=20`** ($\approx 0.10\text{–}0.15\text{ m/pixel}$ GSD, \(\approx 38 \times 38\text{ m}\) ground footprint).
  * **Why $z=20$ is Optimal**: All models show a clear empirical performance peak at $z=20$. At this ground scale, physical ground features (single building footprints, street widths, parcel borders) align directly with the receptive field of the neural network. Spatial coherence and distance-decay curves reach their maximum contrast.
  * **Scale Robustness across Models**:
    * **V3**: Highly sensitive to scale. Calibrated strictly for $z=20$ footprints ($\approx 38\text{ m}$). On coarser scales ($z \le 18$), an entire neighborhood block is packed into a single tile, creating high-frequency texture noise unless processed via quadtree sub-patching.
    * **V11 & V12**: Substantially more robust against scale shifts. The dual-tower architecture and relational self-attention over the token grid preserve macro-zone clustering and spatial boundaries even across coarser scales ($z=16\text{–}19$).
  * **Scale Consistency Rule**: Any consistent scale works, but **all queries and database tiles must share the exact same ground scale**. Do not mix different zoom levels within one index.

---

## Model Variations

| Model | Native Size | Throughput (GPU) | Architecture | Description |
|:---|:---:|:---:|:---|:---|
| **V12** (`Model.v12()`) | $256 \times 256$ px | ~72 tiles/s | Dual-Tower + Relational Transformer | **Production default.** Refined topological regularization; sharpest road network delineation. |
| **V11** (`Model.v11()`) | $256 \times 256$ px | ~76 tiles/s | Dual-Tower + Relational Transformer | Factorized structure and fine detail towers with relational self-attention. |
| **V3** (`Model.v3()`) | $128 \times 128$ px | **~216 tiles/s** | ResNet-18 Baseline | Lightweight, fast visual encoder. Calibrated for $z=20$; high material/vegetation fidelity. |

All models output 256-dimensional unit vectors ($\|z\|_2 = 1.0$).

---

## Code Examples

### 1. Encoding & Similarity API

```python
from geoembed import Model

model = Model.v12(device="gpu", workers=4)

# Single tile -> np.ndarray of shape (256,)
vector = model.encode("tile.png")

# Batch encoding -> np.ndarray of shape (N, 256)
vectors = model.encode(tile_paths, batch_size=32)

# Streaming generator -> yields batches of shape (B, 256)
for batch in model.iter_encode(tile_paths, batch_size=32):
    print(batch.shape)  # (32, 256), dtype=float32

# Cosine similarity -> float in [-1.0, 1.0]
sim = model.similarity("tile_a.png", "tile_b.png")
```

### 2. Continuous Latent Field PCA Raster Map

Project tile embeddings directly into 2D continuous RGB maps with percentile contrast stretching:

```python
# Returns PIL.Image.Image (or np.ndarray if as_image=False)
pca_map = model.render_pca_grid(
    images=tile_paths,
    coords=tile_coords,  # List of (x, y) spatial tile coordinates
    as_image=True,
)
pca_map.save("stuttgart_pca.png")
```

### 3. Multi-Scale Zoom Resampling & Super-Resolution

The models are calibrated for native **$z=20$** ground footprints ($\approx 0.15\text{ m/px}$, $\approx 38\text{ m}$ per tile). When ingesting coarser imagery (e.g. $z=18$, $\approx 150\text{ m}$ footprint), evaluating full tiles directly causes scale aliasing. geoembed resolves this via **quadtree sub-patching**:

```python
# Mode A: Super-Resolution (z18 input -> 16x finer z20 latent field)
# Slices each z18 tile into 16 sub-crops at native z20 footprint,
# expanding the output into a 4x4 coordinate grid.
pca_superres = model.render_pca_grid(
    images=tiles_z18,
    coords=coords_z18,
    input_zoom=18,
    target_zoom=20,
    output_zoom=20,
)

# Mode B: Noise-Free Pooled Macro Parcels (z18 -> z20 -> pooled z18)
# Evaluates sub-crops at native scale, then pools vectors back to 1 per tile.
# Eliminates high-frequency texture noise across regional maps.
pca_clean = model.render_pca_grid(
    images=tiles_z18,
    coords=coords_z18,
    input_zoom=18,
    target_zoom=20,
    output_zoom=18,
)
```
