Metadata-Version: 2.4
Name: env-able
Version: 0.9.2
Summary: AI-ready GIS toolkit for energy and subsurface workflows.
License: MIT
License-File: LICENSE
Requires-Python: >=3.8
Requires-Dist: geopandas
Requires-Dist: openpyxl
Requires-Dist: pyogrio
Requires-Dist: xlrd
Provides-Extra: atlas
Requires-Dist: fastapi; extra == 'atlas'
Requires-Dist: python-multipart; extra == 'atlas'
Requires-Dist: uvicorn; extra == 'atlas'
Provides-Extra: databricks
Requires-Dist: databricks-sql-connector; extra == 'databricks'
Provides-Extra: fast
Requires-Dist: duckdb; extra == 'fast'
Requires-Dist: requests; extra == 'fast'
Provides-Extra: fiona
Requires-Dist: fiona; extra == 'fiona'
Description-Content-Type: text/markdown

# env-able

An open source spatial analysis library built for AI-driven GIS workflows. Designed to give AI systems like Claude reliable, hallucination-free tools for spatial operations in energy and subsurface contexts.

## Install

```bash
pip install env-able

# with Databricks support
pip install env-able[databricks]

# with interactive map server (Atlas)
pip install env-able[atlas]

# with large-scale streaming pulls (DuckDB)
pip install env-able[fast]
```

## Usage

```python
import env_able as env
```

---

## Functions

### `env.pull(table, output=None, wkt_col=None, chunk_size=None, crs="EPSG:4326")`

Pull a Databricks table or query to a local file, chunking around the row/byte limit automatically.

Databricks caps result sets (~4 096 rows for narrow tables, fewer when WKT columns are present). `pull()` paginates with `LIMIT`/`OFFSET`, assembles the complete dataset in memory, and writes it to disk in the requested format.

**Parameters**
- `table` — fully-qualified table name (`catalog.schema.table`) or a complete `SELECT` query
- `output` — destination file path; format inferred from extension (`.gpkg`, `.geojson`, `.shp`, `.parquet`, `.csv`, `.xlsx`). Omit to return a GeoDataFrame or DataFrame.
- `wkt_col` — column containing WKT geometry strings. Auto-detected if omitted.
- `chunk_size` — rows per Databricks query. Defaults to 512 (WKT) or 4 096 (tabular). Reduce if you hit payload errors on wide tables.
- `crs` — CRS to assign geometry. Default `EPSG:4326`.

**Environment variables required**
```
DATABRICKS_HOST       # https://adb-<workspace-id>.azuredatabricks.net
DATABRICKS_TOKEN      # personal access token
DATABRICKS_HTTP_PATH  # /sql/1.0/warehouses/<warehouse-id>
```

```python
import env_able as env

# pull a full table to GeoPackage
env.pull("catalog.schema.wells", "wells.gpkg")

# pull with a filter query
env.pull("SELECT * FROM catalog.schema.wells WHERE state = 'TX'", "wells_tx.gpkg")

# pull tabular (no geometry)
env.pull("catalog.schema.formations", "formations.csv")

# return in-memory without writing
gdf = env.pull("catalog.schema.leases", wkt_col="geom_wkt", crs="EPSG:4269")
```

---

### `env.Clip(input_layer, clip_layer, output=None, where=None, preview=False)`
Clips input features to the extent of a polygon boundary (`ogr2ogr -clipsrc` engine — streaming, handles files larger than RAM). Returns a `SpatialResult`.
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
- `clip_layer` — polygon clip boundary (file path or GeoDataFrame)
- `output` — output file path (`.gpkg`, `.shp`, `.geojson`, etc.)
- `where` — optional SQL attribute filter applied before clipping, e.g. `"STATE = 'TX'"`
- `preview` — if `True`, pushes result to a running Atlas server on completion

```python
result = env.Clip("wells.shp", "texas.gpkg", output="wells_tx.gpkg")
result = env.Clip(gdf, "counties.gpkg", where="STATE = 'TX'")
result.preview("Texas Wells", color="#F68D2E")   # push to Atlas
result.to("wells_tx.geojson")                    # write extra format
print(result)                                    # stats: input, output, dropped, CRS, time
gdf = result.gdf                                 # access as GeoDataFrame
```

---

### `env.Buffer(input_layer, distance, unit="meters", output=None, where=None, preview=False)`
Buffers input features by a given distance (OGR Python API engine). Auto-selects the best UTM zone for accurate metric distances; result is returned in the original CRS. Returns a `SpatialResult`.
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
- `distance` — numeric buffer distance
- `unit` — `meters`, `km`, `miles`, `feet`, `usfeet`, `nautical miles`
- `output` — output file path
- `where` — optional SQL attribute filter
- `preview` — if `True`, pushes result to Atlas on completion

```python
result = env.Buffer("wells.shp", 1, "miles", output="wells_1mi.gpkg")
result = env.Buffer(gdf, 500)                    # 500 m, no file written
print(result)                                    # SpatialResult stats
```

---

### `env.Intersect(input_layer, intersect_layer, output=None, where=None, preview=False)`
Geometric intersection of two layers (`OGR Layer.Intersection()` engine). Input geometries are trimmed to their overlap; attributes from **both** layers appear in the result. Returns a `SpatialResult`.
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
- `intersect_layer` — polygon boundary to intersect against
- `output` — output file path
- `where` — optional SQL attribute filter on the input layer
- `preview` — if `True`, pushes result to Atlas on completion

```python
result = env.Intersect("wells.shp", "permits.gpkg", output="wells_permits.gpkg")
result.preview("Permitted Wells")
```

---

### `env.load(source)` — fluent spatial pipeline
Create a chainable `SpatialPipeline` from a source layer. Add operations with `.buffer()`, `.clip()`, `.intersect()`, set outputs with `.to()`, push to Atlas with `.preview()`, then execute with `.run()` or `await .run_async()`. Intermediate temp files are cleaned up automatically.

```python
result = (
    env.load("wells.shp")
        .buffer(1, "miles")
        .clip("texas.gpkg")
        .to("wells_1mi_tx.gpkg")
        .preview("Wells in TX", color="#F68D2E")
        .run()
)

# Fan-out multiple pipelines in parallel
import asyncio
results = await asyncio.gather(
    env.load("wells.shp").clip("texas.gpkg").run_async(),
    env.load("wells.shp").clip("new_mexico.gpkg").run_async(),
)
```

---

### Async variants
All three operations have async counterparts that run in a thread-pool executor. GDAL releases the GIL so multiple calls run truly in parallel via `asyncio.gather()`.

```python
import asyncio

# Single async call
result = await env.clip_async("wells.shp", "texas.gpkg")

# Fan-out in parallel
r1, r2, r3 = await asyncio.gather(
    env.clip_async("wells.shp", "texas.gpkg"),
    env.buffer_async("wells.shp", 1, "miles"),
    env.intersect_async("wells.shp", "permits.gpkg"),
)
```

---

### `SpatialResult`
Returned by `Clip`, `Buffer`, `Intersect`, and the pipeline `.run()`. Exposes operation stats and chainable output methods.

| Attribute / Method | Description |
|---|---|
| `input_count` | Feature count of the input layer |
| `output_count` | Feature count of the result |
| `dropped` | `input_count - output_count` |
| `slivers` | Polygon features with suspiciously small area (intersection artifacts) |
| `crs` | CRS of the output (e.g. `EPSG:4326`) |
| `elapsed_s` | Wall-clock seconds for the operation |
| `warnings` | List of non-fatal warnings (CRS reprojection, empty result, etc.) |
| `.gdf` | Load result as a GeoDataFrame (lazy, cached) |
| `.preview(name, color)` | Push to a running Atlas server |
| `.to(*paths)` | Write to one or more additional file formats |

---

### `env.morph(input_path, output_path, **kwargs)`
Universal format translation. Converts between shp, gpkg, gdb, csv, xlsx, xls, dbf, geojson, json with automatic CRS handling, field name fixes, and multi-layer support.

- `input_path` — source file or geodatabase
- `output_path` — destination file. Extension sets the format. Use trailing `/` for directory output (one file per layer). Use dot notation for named layers: `roads.parcels.gpkg`
- `x_col`, `y_col` — column names for X/Y coordinates (auto-detected if not provided)
- `wkt_col` — column containing WKT geometry (auto-detected if not provided)
- `crs` — coordinate reference system e.g. `EPSG:4326` (required for tabular → spatial)

```python
env.morph("roads.shp", "roads.gpkg")
env.morph("county.gdb", "county.gpkg")
env.morph("county.gdb", "output_folder/")
env.morph("owners.csv", "owners.geojson", crs="EPSG:4269")
env.morph("owners.csv", "owners.shp", x_col="LONGITUDE", y_col="LATITUDE", crs="EPSG:4269")
env.morph("roads.gpkg", "roads.parcels.gpkg")
env.morph("data.json", "data.gpkg")

# async variant
await env.morph_async("roads.shp", "roads.gpkg")
```

**Smart behavior:**
- GDB / GPKG with multiple layers → detects all layers automatically
- CRS mismatch → auto-reprojects
- Shapefile field name limit (10 chars) → auto-truncates with warnings
- Invalid output path → plain English error
- Empty layers → skipped with a warning, not a crash

---

## env.atlas — interactive map server

`env.atlas` launches a browser-based interactive map (MapLibre GL JS) that Claude can load data into and control programmatically. The user opens it in their browser and fine-tunes from there.

Requires `pip install env-able[atlas]`

```python
import env_able as env

# Start the server (non-blocking — runs in background thread)
env.atlas.serve(block=False)

# Connect and operate
client = env.atlas.connect()
client.add_layer(gdf, "Wells", color="#f5a623")   # push a GeoDataFrame
client.set_viewport([-103.0, 32.0], zoom=7)       # frame the view
print(client.state())                              # check what's on the map
client.save_layout("wells_map.atlas.json")         # save for later
```

**AtlasClient methods:**

| Method | Description |
|---|---|
| `add_layer(data, name, color)` | Push GeoDataFrame or file path; serializes inline, no temp file |
| `upload_layer(path, name, color)` | Load any format (gpkg, geojson, csv, xlsx, zip/shp); converts via morph |
| `remove_layer(name)` | Remove a layer by name |
| `clear()` | Remove all layers |
| `set_layer_color(name, color)` | Change a layer's color; browser updates on next poll |
| `reorder_layers(names)` | Set rendering order — first name = top of map |
| `validate_join(input_layer, input_field, join_source, join_field)` | Preview join match stats without modifying any layer |
| `join_field(input_layer, input_field, join_source, join_field, fields=None)` | Left-join attributes from a loaded layer or table into another layer; `fields` limits which columns are added |
| `set_viewport(center, zoom)` | Set map view — browser flies there within ~2 s |
| `get_viewport()` | Read current viewport (reflects user pan/zoom) |
| `state()` | Layer count, names, colors, feature counts, current viewport |
| `save_layout(path)` | Write full map state to `.atlas.json` |
| `load_layout(path)` | Restore a saved layout |
| `export_svg(path)` | Export all data layers as a vector SVG (no basemap) |
| `is_running()` | Health check |

The browser UI includes an ArcGIS Pro-style ribbon with basemap switching, file upload, and PNG/PDF export, plus a layer panel with drag-and-drop reorder, color picker (server-synced), visibility toggle, zoom-to, and remove.

---

## env.stream — large-scale Databricks pulls

`env.stream` pages arbitrarily large tables through DuckDB without holding them in RAM, writing directly to GPKG, GeoJSON, Parquet, or CSV. Bypasses the ~4096-row / ~2 MB Databricks response cap.

Requires `pip install env-able[fast]`

```python
from env_able.stream import pull_to_file, connector_arrow_frames

# Stream a full table to GeoPackage via Arrow (no row cap)
frames = connector_arrow_frames(
    "SELECT * FROM catalog.schema.wells",
    host="https://adb-xxxx.azuredatabricks.net",
    http_path="/sql/1.0/warehouses/xxxx",
    token="dapixxxx"
)
rows = pull_to_file(frames, "wells.gpkg", wkt_col="geom_wkt")
print(f"{rows:,} rows written")
```

**Transports:**

| Function | Method | Cap |
|---|---|---|
| `connector_arrow_frames` | Databricks SQL connector Arrow batches | None |
| `rest_external_links_frames` | Statement Execution API, Cloud Fetch | None |
| `keyset_frames` | Seek/keyset pagination | Configurable page size |
| `offset_frames` | LIMIT/OFFSET pagination | Configurable page size |

---

## Changelog

### v0.9.0 — 2026-07-21
**Breaking:** `Clip`, `Buffer`, `Intersect` now return `SpatialResult` instead of `GeoDataFrame`. Use `.gdf` to get the underlying GeoDataFrame.

- **GDAL/OGR spatial engine** — operations rewritten as true GDAL calls, not geopandas wrappers
  - `Clip` — `ogr2ogr -clipsrc`; streaming, no full memory load
  - `Buffer` — OGR Python API with auto-UTM zone selection; result in original CRS
  - `Intersect` — `OGR Layer.Intersection()`; attributes from both layers in result
- **`SpatialResult`** — rich return type: counts, CRS, timing, warnings, `.gdf`, `.preview()`, `.to()`
- **Async variants** — `clip_async()`, `buffer_async()`, `intersect_async()` via thread-pool; GDAL releases GIL for true parallelism
- **Fluent pipeline** — `env.load(source).buffer(...).clip(...).to(...).preview(...).run()` and `.run_async()`
- **`where=` filter** — SQL attribute filter on all three operations
- **`morph` spatial→spatial via ogr2ogr** — file-to-file conversions no longer load full dataset into memory
- **`morph_async()`** — async variant of `env.morph`

### v0.8.4 — 2026-07-21
- **`client.join_field()`** — programmatic left-join: carry attributes from any loaded layer or table into another layer in-place; optional `fields` list to limit what's added
- **`client.validate_join()`** — preview match stats without modifying any layer
- **`client.export_svg(path)`** — export all data layers as a vector SVG; browser Export ribbon SVG button
- **Join Field UI** — field checkboxes with Select All / Deselect All; ArcGIS Pro-style two-section validation stats; many-to-one join handling

### v0.8.0 — 2026-07-17
- **Multi-file shapefile upload** — select `.shp` + companions together; missing `.shx` regenerated automatically
- **Layer right-click menu** — Rename, Zoom To, Attribute Table, Remove
- **Folder drag-and-drop** — drag a shapefile folder from Explorer onto the map
- **CRS prompt modal** — manual EPSG/WKT override when CRS cannot be detected
- **ESRI WKT fallback** — handles non-standard projection names via pyproj → GDAL → regex parameter extraction

_Full history in [CHANGELOG.md](CHANGELOG.md)_
