Metadata-Version: 2.4
Name: openmapstack
Version: 0.4.0
Summary: Validate, run, and inspect reproducible OpenMapStack projects
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jsonschema<5,>=4
Requires-Dist: PyYAML<7,>=6
Provides-Extra: geo
Requires-Dist: duckdb>=1.2; extra == "geo"
Provides-Extra: visual
Requires-Dist: playwright>=1.40; extra == "visual"
Provides-Extra: postgis
Requires-Dist: psycopg[binary]>=3.1; extra == "postgis"
Provides-Extra: bigquery
Requires-Dist: google-cloud-bigquery>=3.11; extra == "bigquery"
Provides-Extra: motherduck
Requires-Dist: duckdb>=1.2; extra == "motherduck"
Provides-Extra: all
Requires-Dist: openmapstack[bigquery,geo,motherduck,postgis,visual]; extra == "all"
Dynamic: license-file

# openmapstack

**Four AI agent skills for geospatial decisions, data discovery, spatial SQL and reproducible analysis projects.**

Install (assume you have git and nodejs with npx already installed):
```bash
npx skills@1.5.26 add jaakla/openmapstack-skills --skill open-map-stack -g
```

OpenMapStack gives your favorite AI agent: Claude Code, Codex, Cursor, OpenCode, PI and 50+ other agents not just a one-off analysis, but a **production workflow** from authoritative data discovery through reusable analysis (pipeline.py) to interactive web (dashboard.html) and GIS deliverables (QGIS project). The workflow becomes **inspectable** and **repeatable** as well-defined projects, defined as a `yaml` file with pinned sources, explicit assumptions and CRS choices, deterministic processing in a python script. It allows also isolated overrides, machine-readable validation, and surfaced provenance.

It is open-first (both data and code-wise) and cloud-native by default, built on shoulders of the awesome Open GIS stack: STAC for discovery; GeoParquet, COG, and PMTiles for storage and delivery; DuckDB and PostGIS for compute; and QGIS, MapLibre, and Martin for presentation. It also teaches to deploy GDAL/OGR, GeoPandas, xarray/rioxarray, PDAL, routing engines, spatial SQL, and pragmatic hosted services when scale or reliability requires them.

## What's in this repo

The 0.4.0 collection has four independently installable skills:

| Skill | Use it for |
|---|---|
| [open-map-stack](skills/open-map-stack/SKILL.md) | Ambiguous or multi-stage GIS work; choose sources, compute, storage and delivery together. |
| [reproducible-gis-project](skills/reproducible-gis-project/SKILL.md) | Compile or maintain the project manifest, canonical pipeline, pinned sources, corrections, validation and reruns. |
| [geospatial-data-discovery](skills/geospatial-data-discovery/SKILL.md) | Find and assess open geospatial data under known requirements. |
| [spatial-sql](skills/spatial-sql/SKILL.md) | Write, review or optimize spatial SQL on an already chosen engine. |

Each skill has a small entry point and local references loaded as needed. The
generalist retains bounded-task support and material-analysis requirements when
installed alone. Optional product help is documented in the installed
[companion reference](skills/open-map-stack/references/companion-skills.md).


### Open Map Stack - general guidance

Includes all the skills.

- [generalist SKILL.md](skills/open-map-stack/SKILL.md) — the skill entry point: triggers, global defaults, format and compute decision matrices, anti-patterns, and a quick triage guide.
- [references/data-sources.md](skills/open-map-stack/references/data-sources.md) - lists OSM, Overture, Sentinel/Landsat, regional portals, STAC catalogs and others.
- [references/services-and-scale.md](skills/open-map-stack/references/services-and-scale.md) - depending on case use local installs or hosted/SaaS services for global-scale basemaps, elevation, routing, geocoding, place search, and postcodes.
- [references/user-data-sources.md](skills/open-map-stack/references/user-data-sources.md) - the user's own warehouse data: credentials by reference, read-only discovery, approval-gated snapshots, and the pin classes that make a warehouse table reproducible.
- [references/formats-and-crs.md](skills/open-map-stack/references/formats-and-crs.md) - how to choose formats, conversions, projections, EPSG codes.
- [references/processing.md](skills/open-map-stack/references/processing.md) - when and how to use GDAL/OGR, GeoPandas, xarray, DuckDB, PostGIS, PDAL and other open geo processing tools.
- [references/analytics.md](skills/open-map-stack/references/analytics.md) — do vector/raster analytics, terrain, hydrology, network, point clouds, geocoding etc.
- [references/web-delivery.md](skills/open-map-stack/references/web-delivery.md) — renderer selection for maps, PMTiles, MVT, Martin, TiTiler, MapLibre, deck.gl, kepler.gl, and lonboard formats and engines.
- [references/qgis.md](skills/open-map-stack/references/qgis.md) — QGIS desktop, plugins, PyQGIS, Processing, QGIS MCP.
- [references/validation-and-ops.md](skills/open-map-stack/references/validation-and-ops.md) — validation, manifests, attribution, and deployment checks, including the machine-readable reproducible-project contract.
- [references/project-spec.md](skills/open-map-stack/references/project-spec.md) — the specific`openmapstack-project/v1` schema: compiling any material analysis into a reproducible GIS project (`project.yaml`, pipeline, source provenance, overrides, validation, semantic presentation, QGIS output).
- [references/project-workflow.md](skills/open-map-stack/references/project-workflow.md) — the mandatory material-analysis workflow and delivery rules, loaded when a task needs a reproducible project.
- [templates/](templates/) — ready scaffolds (`project.yaml`, `pipeline.py`, `presentation.yaml`, `validation.yaml`) for new projects.

### Reproducible GIS project
Well-defined GIS project manifest and pipeline skills.

### Geospatial data discovery
Geodata discovery skills - open, where possible authoritative source data for global and local coverage.

### Spatial SQL
Geospatial SQL specific skills only.


### Additional materials:

- [examples/tartu-development/](examples/tartu-development/) — a fully-worked reproducible project matching the acceptance scenario: source provenance + timestamps, explicit assumptions, two verified project overrides (a scenario attribute change with prior-value verification, and hypothetical scenario geometry), deterministic pipeline, machine-readable validation, and semantic presentation.
- [evals/](evals/) — the eval suite grading whether an agent reaches the right analytical answer, respects the GIS-method guardrails, and reruns reproducibly, with the `openmapstack-project/v1` contract as the substrate that makes those independently checkable: `python evals/run.py --mode fixture` runs deterministic, no-LLM checks against real generated artifacts (analytical correctness against known geospatial truth, metric CRS, source immutability, schema, overrides, validation integrity, presentation contract, and clean reruns), plus adversarial cases and a pluggable live-agent benchmark (Claude Code, Codex, and any OpenAI-compatible API such as OpenRouter — URL and model via `OPENAI_COMPATIBLE_*` env, API key as a secret).
- [`openmapstack/`](openmapstack/) — the installable `openmapstack validate/run/inspect` CLI for auditing and executing `openmapstack-project/v1` projects, plus [`openmapstack/checks/`](openmapstack/checks/): the reusable, semantic check library. All but five of its checks are oracle-free, so the same functions that grade the eval suite also grade a user's own project on data this repository has never seen.
- [docs/openmapbench-interop.md](docs/openmapbench-interop.md) — the narrow, versioned contract a benchmark harness such as OpenMapBench consumes: `openmapstack checks` / `check` / `api-info` (`openmapstack-check-api/v1`), the packaged result schemas, skill snapshots, arm provenance, and exported task bundles.
- [`.claude-plugin/`](.claude-plugin/) — Claude Code plugin and marketplace manifests, so the repository can also be installed with `/plugin install`. Validated in CI by [`.github/workflows/plugin.yml`](.github/workflows/plugin.yml).

Some my local Estonia-specific guidance (Maa- ja Ruumiamet, ETAK, EPSG:3301 / L-EST97) is included for convenience. But most of the major global sources are included for world-wide coverage.

## Install

The 0.4.0 collection provides four independently installable skills and a
separately installed, matching CLI. From a checkout, install the generalist:

```bash
npx skills@1.5.26 add . --skill open-map-stack -a codex -y
python -m pip install '.[geo]'
```

Use `-a claude-code` for Claude Code, `-g` for global installation and `--copy`
for independent copies. Select additional skills by repeating `--skill NAME`;
`--skill '*'` selects the full collection. Each skill includes local templates,
project schema, a trimmed worked Tartu example and CLI setup instructions.
No installed sibling is required. Generated example outputs and downloaded
source data are omitted; its pipeline needs network access and its documented
GIS environment.

Once the release tag and Python package are published, pin both parts of the
coordinated release:

```bash
npx skills@1.5.26 add https://github.com/jaakla/openmapstack-skills/tree/v0.4.0/skills/open-map-stack -a codex -y
python -m pip install 'openmapstack[geo]==0.4.0'
```

For a floating install use `npx skills@1.5.26 add jaakla/openmapstack-skills
--skill open-map-stack`. Check installed skills with `npx skills@1.5.26 list`,
update with `npx skills@1.5.26 update open-map-stack --project`, and remove with
`npx skills@1.5.26 remove open-map-stack`. Use `--global` instead of `--project` for a global update; match the scope for list/remove too.
Updating a floating install advances its version; release-pinned installs
should be replaced with an explicitly chosen release.

### Claude Code plugin

```text
/plugin marketplace add jaakla/openmapstack-skills
/plugin install open-map-stack@open-map-stack
```

The plugin retains its identity and discovers `skills/<name>/SKILL.md`.
For local development use `claude --plugin-dir /path/to/checkout`; metadata
validation and component inventory require no paid model run.

### Manual installation and migration

Copy `skills/<name>/` into your agent's skill directory. Do not copy the whole
collection checkout into a single skill folder. If replacing a 0.3.0 root
installation, retain the `open-map-stack` name and replace its installed payload;
do not keep both root and nested copies. Inspect your actual installer lockfile
and scope before changing anything. Legacy `open-gis` is a different identifier:
remove it explicitly if it is an unwanted duplicate, rather than silently
rewriting its lock entry.

The CLI is separate from skill installation. Verify `openmapstack --version`
against the installed skill's `metadata.version`. Before publication, a wheel
or pinned Git commit from the matching checkout is the supported alternative.

## Use

Agents discover the installed skill descriptions and select relevant guidance.
Selection depends on the agent; explicitly naming a skill can help when you
want a particular owner. Installing the collection does not require loading
all four skills for every request.

- “We need regional analysis and browser maps for a billion building records; choose the architecture.” → `open-map-stack`.
- “Keep this analysis and stack, but make it reproducible and auditable.” → `reproducible-gis-project`.
- “Find authoritative Estonian building footprints; explain coverage, license and a reproducible pin.” → `geospatial-data-discovery`.
- “Review this PostGIS query for a 500-metre distance test on SRID 4326 geometries.” → `spatial-sql`.

The primary skill can use focused support. A bounded lookup or query review
does not require a project; material multi-stage analyses retain the complete
reproducibility contract. Casual place lookups and ordinary non-spatial coding
are outside the collection's scope.

See [0.4.0 release preparation](docs/release-0.4.0.md) for executed installation
checks, consumer migration and remaining release acceptance.

## Project CLI

> The key innovation of the skill is not just do the work every time again and then forget it, but to create special well-defined project with data and process descriptions and rerunnable scripts, so the whole process becomes investigatable and repeatable.

To help with that we have special CLI to work with the projects.

The CLI operates on an `openmapstack-project/v1` manifest. A project directory may
be supplied in place of its `project.yaml` file.

```bash
# Audit the complete artifact, including outputs, report, and run record.
openmapstack validate path/to/project.yaml

# Check the produced artifacts without requiring a golden answer.
openmapstack verify path/to/project.yaml

# Run the one canonical pipeline, then validate what it produced.
openmapstack run path/to/project.yaml

# Review sources, versions, overrides, ordered steps, outputs, and latest run.
openmapstack inspect path/to/project.yaml

# Copy SKILL.md, references/, and templates/ into a hashed, inspectable snapshot.
openmapstack skill-snapshot --out /tmp/oms-skill --json
openmapstack skill-snapshot --inspect /tmp/oms-skill

# Read-only discovery of a warehouse source, then an approval-gated snapshot.
openmapstack source discover path/to/project.yaml --source parcels
openmapstack source snapshot path/to/project.yaml --source parcels \
  --query "SELECT id, geom FROM cadastre.parcels" --destination data/source/parcels.parquet --approve
```

Useful automation options:

```bash
openmapstack validate project.yaml --json --output validation/cli-report.json
openmapstack validate project.yaml --strict       # warnings also return non-zero
openmapstack validate project.yaml --preflight    # skip not-yet-generated artifacts
openmapstack run project.yaml --dry-run        # print the command, execute nothing
openmapstack run project.yaml --json
openmapstack inspect project.yaml --json
```

### Sampled runs — nail it before you scale it

A wide-area analysis can run for hours before a late step fails. A sampled run
executes the same pipeline over a deliberately smaller slice, so failure
arrives in minutes:

```bash
openmapstack run project.yaml --sample                     # the manifest's declared sample
openmapstack run project.yaml --sample-area 26.6,58.3,26.8,58.4
openmapstack run project.yaml --sample-rows 5000
openmapstack run project.yaml --sample-fraction 1.0
```

Each flag binds a `runtime.implementation.parameters` entry that declares the
matching `role`; sampling a project that declares none is refused, naming what
the manifest must add. The canonical run still passes nothing.

**A sampled run proves the pipeline executes; it does not establish the
result.** Clipping to a test AOI breaks neighbourhood operations at the cut and
row sampling destroys the spatial coherence a join needs, so sampled counts are
not answers. That is enforced, not merely advised: a sampled run record is
marked `mode: sampled`, must record what it *realized* rather than only what
was requested, and can never become `runs.latest` — `openmapstack validate`
reports this as `runs.sample_isolation`, and `run --sample` fails outright if a
pipeline promotes its own sampled run — by moving `runs.latest`, by rewriting
the record it already points at, or by leaving no sampled record behind at all.
See `references/project-spec.md`.

### `openmapstack verify` — check the analysis, not just the paperwork

`validate` audits the manifest and its bookkeeping. `verify` runs the check
library in `openmapstack/checks/` against what the pipeline actually produced:
geometry read back through DuckDB Spatial, dataset CRS read from the artifact
rather than the manifest's claim, validation evidence recomputed from the
geodata it summarises, and QGIS project structure and runtime loading where
PyQGIS is available.

```bash
openmapstack verify path/to/project.yaml
openmapstack verify path/to/project.yaml --rerun     # + rebuild from source and compare
openmapstack verify path/to/project.yaml --metamorphic   # + run declared no-oracle relations
openmapstack verify path/to/project.yaml --json --output validation/verify-report.json
openmapstack verify path/to/project.yaml --strict    # warnings and not-testable also return 1
```

These checks require no repository-owned golden answer, so they work on data
neither this repository nor the model has seen. They establish bounded
structural, provenance, artifact, and reproducibility predicates; they do not
prove every project-specific analytical answer.

`--rerun` is the strongest signal available without a known answer. It rebuilds
the project in an empty workspace from only the manifest, the declared
immutable inputs, and the declared dependencies, runs the one canonical
entrypoint, re-hashes the sources, and compares the outputs semantically. A
pipeline that cannot reproduce itself, or that mutates its own declared
immutable inputs, is not trustworthy whatever its numbers say.

The check plan is derived from the manifest rather than configured, so a
project cannot opt out of a check by omitting it: a declared output is a
checked output. A check whose dependency is missing reports `not_testable` and
is counted separately — never a silent pass. A mixture of executed and
`not_testable` checks has aggregate status `warning`, and every report includes
`applicable`, `executed`, and `execution_rate` coverage. Install
`openmapstack[geo]` for the DuckDB-backed geodata checks; PyQGIS comes from a
system QGIS install.

See [the applicability reference](docs/verify-applicability.md) for the exact
plan conditions, dependencies, current regression evidence, and deliberate
exclusions. In particular, browser/dashboard checks are not yet part of the
automatic `verify` plan.

Project-specific known answers can be declared under
`validation.expectations[]`. The five allowlisted checks cover row count,
feature presence/absence, one feature-field value, and field range. New
expectations start as `attestation.status: unverified`; they produce a warning
and are not executed. The JSON report supplies the exact
`expected_expectation_sha256` an independent reviewer must bind, together with
the current `runs.latest.inputs_hash`. Changing the expected check, arguments,
inputs, or a retained local evidence file invalidates the attestation and
returns it to warning status. See
[the project contract](skills/open-map-stack/references/project-spec.md#26-validation).

Where no golden answer exists at all, `validation.metamorphic[]` declares
relations that must hold under a controlled perturbation: shuffle a source and
the result must not change, duplicate every feature and a keyed set must not
change, widen an inclusion buffer and no candidate may disappear. Each relation
states the precondition that makes it valid, is executed by
`verify --metamorphic` in an isolated copy against the project's own pipeline,
and reports `not_testable` with the reason when the precondition does not hold
on the actual data. See [the project contract](skills/open-map-stack/references/project-spec.md#26-validation).

`openmapstack source` is the connector pilot for the user's own data
(DuckDB local files and PostGIS). Credentials are referenced, never stored;
discovery is read-only with a statement timeout; a snapshot is a dry run
until `--approve`, is limited by rows and bytes, lands only under
`data/source/`, and hands back the `pin` block that makes the source
reproducible. A warehouse table with only a timestamp is not pinned; an
expired backend snapshot is reported as `not_reproducible`. See
[user data sources](skills/open-map-stack/references/user-data-sources.md).

`validate` checks manifest structure, source retrieval/version/licensing data,
CRS declarations, processing graph resolution, override provenance and files,
output existence, validation-report parity/status propagation, override
application results, and run-record identity/hashes. GIS-specific checks such as
geometry validity remain the pipeline's responsibility; the CLI verifies that
each declared check appears exactly once with an explicit result.

Normal validation warnings return exit code 0 so known limitations remain
representable. Failures return 1; malformed invocation or an unstartable runtime
returns 2. `--strict` makes warnings return 1.

## What this skill will and won't do

**Will:**
- Recommend modern, cloud-native formats (GeoParquet, COG, PMTiles) and flag legacy patterns (Shapefile output, MBTiles for new deployments).
- Push spatial joins to DuckDB / PostGIS instead of Python loops.
- Discover data via STAC before downloading.
- Preserve license metadata (OSM ODbL, Overture per-source, Sentinel attribution).
- Pin dataset versions for reproducibility (Overture releases, STAC item IDs, OSM extract dates).
- Compile material multi-stage analysis into a reproducible GIS project (`project.yaml` + pipeline + overrides + validation), deriving the final map/dashboard from it.

**Won't:**
- Trigger on simple location lookups ("what city is this?") or casual map references with no analytical work.
- Default to proprietary services when an open/self-hosted option fits the scale, quality, privacy, and budget.

## License

Licensed under the [MIT License](LICENSE).

## Contributing

Issues and PRs welcome at [github.com/jaakla/openmapstack-skills](https://github.com/jaakla/openmapstack-skills). When adding a new tool or workflow, place it in the matching reference file and add a one-row entry to the relevant decision matrix in [generalist SKILL.md](skills/open-map-stack/SKILL.md).
