Metadata-Version: 2.5
Name: streamlit-lonboard
Version: 0.2.1
Summary: Streamlit component for lonboard: fast GeoArrow/deck.gl maps without GeoJSON
Project-URL: Homepage, https://github.com/relativityhd/streamlit-lonboard
Project-URL: Repository, https://github.com/relativityhd/streamlit-lonboard
Project-URL: Issues, https://github.com/relativityhd/streamlit-lonboard/issues
Project-URL: Changelog, https://github.com/relativityhd/streamlit-lonboard/blob/main/CHANGELOG.md
Author: Tobias Hoelzer
License-Expression: MIT
License-File: LICENSE
Keywords: deckgl,geoarrow,geospatial,lonboard,streamlit
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Visualization
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: ipywidgets<9,>=8
Requires-Dist: lonboard>=0.10
Requires-Dist: pyarrow!=25.0.0,>=14
Requires-Dist: streamlit>=1.59
Provides-Extra: bench
Requires-Dist: playwright; extra == 'bench'
Requires-Dist: pydeck; extra == 'bench'
Requires-Dist: zarr>=3; extra == 'bench'
Provides-Extra: dev
Requires-Dist: geopandas>=1; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff<0.16,>=0.15; extra == 'dev'
Description-Content-Type: text/markdown

# streamlit-lonboard

A Streamlit custom component for [lonboard](https://github.com/developmentseed/lonboard) — fast, GPU-accelerated geospatial visualization in Streamlit, powered by [deck.gl](https://deck.gl) and [GeoArrow](https://geoarrow.org).

> **AI Slop** This library was completely generated by the Claude Code Agent utilizing the latest models (Sonnet 5, Opus 5, and Fable 5). I needed this connector for another project, where I also tested its functionality. Every single line of code was reviewed by myself; overall, the code looked okay-ish to me. Nothing I would do myself, but it works and it works well enough. Of course, I would welcome proper first-party support by Lonboard to make this package obsolete. Until then, this slop may allow us to use Lonboard for Streamlit dashboards. If you hesitate to use AI-generated code, do NOT use this library!
>
> **Status: early development.** Scatterplot/Path/Polygon/SolidPolygon/Column/PointCloud layers, multi-layer maps, click/hover picking, and view-state persistence across reruns all work. Heatmap is wired but untested. DGGS layers — [H3](https://developmentseed.org/lonboard/latest/api/layers/h3-hexagon-layer/), [S2](https://developmentseed.org/lonboard/latest/api/layers/s2-layer/), [A5](https://developmentseed.org/lonboard/latest/api/layers/a5-layer/), Geohash — and Arc layers work too, but they carry geometry in accessor columns (cell IDs / point pairs) rather than a bounding geometry column, so lonboard can't auto-compute a default view — pass an explicit `view_state=` (H3 *can* auto-center, but only if [`h3-py`](https://pypi.org/project/h3/) is installed). Trip layers render a single static frame (drive `layer._current_time` yourself, e.g. from a slider, for animation — no built-in animation loop). Text/Bitmap/BitmapTile/Raster layers aren't supported yet (Text is provisional upstream; Bitmap/Raster carry raster, not GeoArrow, data).
>
> All four lonboard **layer extensions** work — `PathStyleExtension`, `DataFilterExtension` (pairs well with `st.slider` driving `layer.filter_range`), `BrushingExtension`, `CollisionFilterExtension`. `st_lonboard()` also forwards `picking_radius`, `parameters`, `use_device_pixels`, `custom_attribution`, `map.controls` (fullscreen/zoom/scale, on by default - `GeocoderControl` isn't supported), and hover `tooltip=` (bool or explicit column list). See `examples/extensions_app.py`.
>
> **pyarrow 25.0.0 is excluded** (`pyarrow>=14,!=25.0.0` in `pyproject.toml`): its bundled mimalloc 3.3.1 segfaults when libarrow is first loaded on a non-main thread that then exits — which is exactly how Streamlit runs every script. Known upstream as [apache/arrow#50471](https://github.com/apache/arrow/issues/50471) / [microsoft/mimalloc#1287](https://github.com/microsoft/mimalloc/issues/1287); no fixed release yet. If another dependency forces 25.0.0 on you, set `ARROW_DEFAULT_MEMORY_POOL=system` as a workaround. All Python versions ≥3.11 (including 3.14) are supported.

## Install

```sh
uv add streamlit-lonboard
```

or via pip

```sh
pip install streamlit-lonboard
```

## Why?

Streamlit's built-in DeckGL support (`st.pydeck_chart`) goes through [pydeck](https://pydeck.gl/), which serializes data as **GeoJSON/JSON** — slow to encode, slow to transfer, slow to parse, and impractical beyond ~100k features.

Lonboard instead moves data as **Apache Arrow** (GeoArrow) binary buffers that deck.gl can consume with zero parsing. But lonboard is built on [anywidget](https://anywidget.dev/) / Jupyter widgets, which Streamlit does not support. The lonboard maintainers consider a Streamlit connector [out of scope for lonboard itself, but support a third-party one](https://github.com/developmentseed/lonboard/discussions/342) — this project is that connector.

The previously suggested workarounds don't cut it:

- `Map.to_html()` + `st.components.v1.html`: static snapshot, no bidirectionality, full re-render on every rerun, huge inlined HTML.
- [`streamlit-deckgl`](https://pypi.org/project/streamlit-deckgl/): pydeck/JSON only — exactly the bottleneck we want to avoid.

## How it works

```
lonboard Map/Layers (Python)          frontend (TypeScript)
  pyarrow.Table (GeoArrow)              apache-arrow: parse IPC
    → Arrow IPC bytes          ──────►    → @geoarrow/deck.gl-layers
  layer props → JSON                       → deck.gl + MapLibre basemap
        ▲                                       │
        └── picking / view state (bidi) ◄───────┘
```

Data crosses the Python↔browser boundary as raw Arrow IPC bytes via Streamlit's [custom components v2](https://docs.streamlit.io/develop/concepts/custom-components/components-v2) — no GeoJSON anywhere in the pipeline.

## API

```python
import geopandas as gpd
import streamlit as st
from lonboard import ScatterplotLayer
from streamlit_lonboard import st_lonboard

gdf = gpd.read_parquet("internet-speeds.parquet")
layer = ScatterplotLayer.from_geopandas(gdf, get_fill_color=[255, 0, 0])

result = st_lonboard(layers=[layer], height=600, key="map")
st.write("Clicked feature index:", result.clicked)
```

See `examples/app.py` for scatterplot/path/polygon,
`examples/dggs_app.py` for H3 and Arc layers (the "geometry lives in
accessor columns" case — see the status note above), and
`examples/extensions_app.py` for layer extensions, `tooltip=`, and map
controls/attribution.

## Performance

Streamlit reruns your whole script on every interaction, so building this
`ScatterplotLayer` from scratch happens again on every rerun unless you cache
it. **Wrap layer construction in `@st.cache_resource`**:

```python
@st.cache_resource
def build_layer():
    gdf = gpd.read_parquet("internet-speeds.parquet")
    return ScatterplotLayer.from_geopandas(gdf, get_fill_color=[255, 0, 0])

layer = build_layer()
result = st_lonboard(layers=[layer], height=600, key="map")
```

This matters more than it might look like: `st_lonboard()` memoizes its own
Arrow serialization keyed on the layer *object*, so a cached layer skips
re-serialization entirely on reruns that don't touch it (invalidated
automatically if you mutate a layer's properties). Without
`@st.cache_resource`, a fresh layer object is built every rerun and the cache
never hits. See `examples/app.py` for a full example and
[`IMPLEMENTATION_PLAN.md`](./IMPLEMENTATION_PLAN.md) Phase 4 for the full
performance investigation, including a genuinely surprising find: Streamlit's
component runtime already skips re-parsing and re-rendering on the frontend
entirely when a rerun's output is byte-for-byte unchanged (see
[`benchmarks/RESULTS.md`](./benchmarks/RESULTS.md) for measured numbers at
10k/100k/1M points) — so the main thing left to optimize is Python-side
re-serialization, which is exactly what the cache above avoids.

### Colour-only updates

Changing just a colormap — the most common dashboard interaction — used to cost
as much as drawing the map from scratch. Streamlit re-sends the whole component
payload whenever anything in it changes, so new colours arrived as a brand-new
Arrow table, and deck.gl treated that as new data: every cell boundary
re-derived, every polygon re-tessellated. At 160k DGGS cells that was seconds of
frozen browser tab.

The frontend now fingerprints each column's contents after parsing, and reuses
the existing tessellation when the geometry-bearing columns are unchanged,
re-uploading only the accessor columns that actually differ. Measured on 160k
high-precision H3 cells: a recolour rerun went from **2.2s of blocked main
thread to 0ms**, with pixel-identical output. It is automatic — no API, no flag.
A genuine geometry change still rebuilds in full.

### Layer construction cost

lonboard layers are ipywidgets `Widget`s, and `Widget.__init__` unconditionally
opens a Jupyter comm — which serializes the entire table and every accessor
column to Parquet to fill a comm-open message. Under Streamlit there is no
kernel, so that message goes straight into a dummy comm and is discarded.

Importing `streamlit_lonboard` therefore patches lonboard's two widget base
classes so this work is skipped (measured: `A5Layer` at 200k rows drops from
62ms to 0.6ms; a `ScatterplotLayer.from_geopandas` at 200k, where the
GeoDataFrame → Arrow conversion is real work, roughly halves). What
`st_lonboard()` puts on the wire is byte-for-byte identical either way — it
reads the layer's traits and does its own Arrow IPC encoding, never touching
lonboard's Parquet path. As a side effect, layers no longer accumulate in
ipywidgets' global `_instances` registry, which nothing drains under Streamlit.

The trade-off: patched widgets have `widget.comm is None` and no `model_id`, so
`Map.to_html()` / `Map.as_html()` raise. Either set the environment variable
below, or give one widget its comm back with
`ipywidgets.Widget.open(widget)` before exporting.

### Environment variables

| Variable | Effect |
| --- | --- |
| `STREAMLIT_LONBOARD_KEEP_WIDGET_COMM=1` | Keep stock ipywidgets behavior (see above). Must be set **before** `import streamlit_lonboard`. |
| `ST_LONBOARD_PERF=1` | Log serialize/pack/mount timings to stderr and emit `st-lonboard:*` marks in the browser's performance timeline. |

### Compression

`st_lonboard(..., compression="auto" | "parquet" | "zstd" | "gzip" | None)`,
default `"auto"`, controls how each layer's Arrow table travels to the
browser. `"auto"` decides **per layer**, by table size:

| layer size | ships as |
| --- | --- |
| under 1 MB | plain, near-zero-copy Arrow IPC |
| 1–20 MB | ZSTD-compressed Arrow IPC |
| 20 MB and up | Parquet (ZSTD + BYTE_STREAM_SPLIT), decoded by `parquet-wasm` |

On real data (an Arctic EO cube on the A5 DGGS, 358k cells) the middle tier
turns a 12.6 MB payload into 2.6 MB — 0.21× — for ~15 ms of Python and
~23 ms of browser decode. Uniform-random synthetic coordinates compress far
worse (0.91×), so **benchmark your own data** rather than either number.

Parquet compresses ~20% better still, but costs roughly 3× the encode and
decode time plus a one-time ~1.8 MB (gzipped, then browser-cached)
`parquet-wasm` download — worth it only when bandwidth clearly dominates,
which is why `"auto"` reserves it for the largest layers.

Forcing a specific mode:

| mode | use it when |
| --- | --- |
| `"zstd"` | you want compression below the 1 MB threshold too |
| `"parquet"` | bandwidth-bound: smallest payload, at ~3× the CPU and a one-time WASM download |
| `"gzip"` | compatibility only — superseded by `"zstd"` on size, encode, and decode |
| `None` | purely local use, where transfer is free and any encode/decode is pure overhead |

Apps whose layers all stay under 1 MB ship byte-identical payloads to
previous versions, and only the top tier ever fetches the WASM. Full
methodology and numbers:
[`benchmarks/RESULTS.md`](./benchmarks/RESULTS.md) (Phases 4h–4k).

### vs. `st.pydeck_chart` and `Map.to_html()`

Measured across 10k-10M points (`benchmarks/playwright_driver.py`,
`benchmarks/payload_sizes.py`; full numbers and methodology in
[`benchmarks/RESULTS.md`](./benchmarks/RESULTS.md)):

- **Wire size**: `st_lonboard`'s Arrow IPC payload is a consistent ~8.3x
  smaller than `st.pydeck_chart`'s JSON at every scale tested (230MB vs.
  1.9GB at 10M points).
- **`Map.to_html()` embedded via `st.components.v1.html`** — the workaround
  people use today without a custom component — **doesn't render at all**,
  at any scale. Root cause: inside Streamlit's sandboxed `srcdoc` iframe,
  `document.location.href` is the opaque string `"about:srcdoc"`, which
  breaks requirejs/anywidget's module-loading URL resolution; the actual
  widget bundle never loads and no error is shown. The same HTML renders
  fine served standalone (outside an iframe).
- **`st.pydeck_chart` itself renders fine interactively**, but rendering
  timing wasn't reliably measurable under headless browser automation in
  our environment (an intermittent WebGL/GPU stall unrelated to pydeck's
  correctness) — reported as an environment limitation rather than forced.

## Development

Managed with [uv](https://docs.astral.sh/uv/). A [Hatchling build
hook](hatch_build.py) runs `npm install && npm run build` automatically
whenever the package is built or synced, so `uv sync`/`uv build` produce a
wheel with the frontend already bundled into
`src/streamlit_lonboard/frontend_dist/` (gitignored source-tree-side; only
Node is required to build it, not to install the published wheel):

```bash
uv sync --extra dev
uv run streamlit run examples/app.py
```

If you edit the frontend, run `cd frontend && npm run dev` (watch build) or
`npm run build` (one-off) yourself and refresh the browser tab — the build
hook only runs when the package itself is (re)built (`uv sync`/`uv build`),
not on every `uv run`.

```bash
uv build          # sdist + wheel into dist/
uv run pytest     # tests/test_serialize.py
uv run ruff check # lint
```

## License

MIT for this project's own code (Python and `frontend/src/`). The built
`frontend_dist/index.js` bundles compiled code from deck.gl, apache-arrow,
maplibre-gl and their transitive dependencies under their own licenses
(mostly MIT/BSD-3-Clause, with Apache-2.0 for apache-arrow and flatbuffers);
a generated `THIRD-PARTY-NOTICES.txt` listing them and their license texts
ships alongside it in every wheel.

## Acknowledgements

- [lonboard](https://github.com/developmentseed/lonboard) by Development Seed
- [@geoarrow/deck.gl-layers](https://github.com/geoarrow/deck.gl-layers)
- Prior discussion: [developmentseed/lonboard#342](https://github.com/developmentseed/lonboard/discussions/342)
