Metadata-Version: 2.4
Name: olmoearth-inferencex
Version: 1.11.0
Summary: A Python package for assessing Earth-observation classification maps. It ranks map windows by model confidence for manual review, compares two maps of the same area, and estimates error rates and per-class accuracy from a labelled sample.
Author: Ziming Qi
License-Expression: Apache-2.0
Project-URL: Documentation, https://olmoearth-inferencex.readthedocs.io
Project-URL: Repository, https://github.com/2imi9/olmoearth_inferenceX
Project-URL: Changelog, https://github.com/2imi9/olmoearth_inferenceX/blob/main/CHANGELOG.md
Keywords: earth-observation,remote-sensing,selective-classification,error-ranking,OlmoEarth,audit
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Operating System :: OS Independent
Requires-Python: <3.14,>=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.26.4
Requires-Dist: pyyaml>=6.0
Requires-Dist: huggingface_hub>=0.20
Provides-Extra: encoder
Requires-Dist: olmoearth-pretrain<0.2,>=0.1; extra == "encoder"
Requires-Dist: torch<2.8,>=2.7; extra == "encoder"
Provides-Extra: geo
Requires-Dist: rasterio>=1.4; extra == "geo"
Requires-Dist: geopandas>=1.0; extra == "geo"
Requires-Dist: shapely>=2.0; extra == "geo"
Requires-Dist: pystac-client>=0.8; extra == "geo"
Requires-Dist: planetary-computer>=1.0; extra == "geo"
Requires-Dist: matplotlib>=3.8; extra == "geo"
Requires-Dist: scipy>=1.11; extra == "geo"
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.26; extra == "mcp"
Dynamic: license-file

olmoearth_inferenceX
====================

olmoearth_inferenceX checks an Earth-observation map after inference: where it is likely wrong, how wrong it is,
which part you can trust, and which of two maps is more accurate. It was built to check OlmoEarth's inference
outputs. It reads only the per-class scores a model writes out, never runs the model and needs no torch, so it works
as well on other models' maps and on published products that ship a confidence layer.

<img src="https://raw.githubusercontent.com/2imi9/olmoearth_inferenceX/main/docs/figures/architecture.png" alt="Structure of olmoearth-inferenceX. Left: Sentinel-2 imagery goes through a model's encoder and decoder to its outputs, a class map and a confidence map. Right: the outputs are pooled to windows. Without reference labels, ranking the windows by confidence gives the review set, and comparing two maps A and B gives the disagreement map. With a labelled sample, a sample of the windows where A and B differ gives their accuracy difference with a 95% interval; a random sample with reference labels gives the error rate with an exact 95% interval and, by exact zone tests, the certified zone" width="900">

Documentation: https://olmoearth-inferencex.readthedocs.io/


Install
-------

```bash
pip install "olmoearth-inferencex[geo]"    # Python 3.11 to 3.13; without [geo] it reads .npy only
oe-inferencex demo                          # a real Dynamic World tile; needs no data
```


Usage
-----

| Question | Commands | Labels |
|---|---|---|
| Where do I look first? | `assess scores.tif --out audit` | none |
| How wrong is the map? | `sample scores.tif --budget 300 --design random --out to_label.csv`, fill in `wrong`, then `estimate to_label.csv` | about 300 random windows |
| Which part can I trust? | `certify to_label.csv --alpha 0.05` | the same windows |
| Which of two maps is more accurate? | `sample a.tif --other b.tif --budget 100 --out pairs.csv`, fill in `reference_class`, then `estimate pairs.csv` | about 100, where the maps differ |
| Where do two maps differ? | `compare a.tif b.tif --out diff` | none |

Every command starts with `oe-inferencex`. The input is the model's per-class scores before the argmax, as a GeoTIFF
or a `.npy` array of shape `(C, H, W)`. `plan` says how many labels a question needs, `decide` turns a result into
yes, no or undetermined, and `--help` lists each command's options.
[Quick start](https://olmoearth-inferencex.readthedocs.io/en/latest/Usage/#quick-start) runs each one on a test map.

Limits: without labels it says where to look, not how wrong the map is. On the one test of a deployed model at a new
year and place, graded at the plots of an independent random sample that had imagery, confidence ranked the errors
with an AUROC of 0.64 under a one-to-one crosswalk, below the preregistered 0.70; that run cannot say whether the year,
the place, the legend or the move from expert-placed points caused it. A model run on inputs unlike its training data can be confident and wrong
(`--condition` treats each input condition on its own). Every interval and certified zone assumes the labels are
right, and labelling more and rerunning is a second test, except on a sequential sample.


Use from an agent
-----------------

`oe-inferencex mcp` is a local MCP server; nothing is hosted. With [uv](https://docs.astral.sh/uv/) installed:

```bash
claude mcp add --scope user oe-inferencex -- uvx --from "olmoearth-inferencex[geo,mcp]" oe-inferencex mcp
```

Use a strong model: in a pilot, Claude Sonnet kept the tools' limits and Claude Haiku often dropped them. To try it,
write the demo tile into an empty folder, start the agent there and ask:

```bash
uvx --from olmoearth-inferencex oe-inferencex demo
```

> Where should I look first in oe_inferencex_demo/sample_probabilities.npy? Use windows of
> 1 pixel, and grade the order against oe_inferencex_demo/sample_truth.npy.


Evidence
--------

Tested on linear probes of sixteen encoders over the tasks of Ai2's embedding suite, and on seven published
land-cover products against six reference samples, with the reference points standing in for the map. On a deployed
OlmoEarth model, a re-run of Ai2's configuration for its fine-tuned AWF model, the confidence ranked the map's errors
with an AUROC of 0.85 at Ai2's expert-placed validation points but 0.64 on 2017 imagery at the plots of an independent
random sample that had imagery (one-to-one crosswalk), below the preregistered 0.70; the record's own fine-tune of the
task on OlmoEarth v1.2 gave the same verdicts (0.81 and 0.64). On a second task, Ai2's Forest Loss Driver model graded
at 2026 MapBiomas Alerta alerts in the Amazon, the confidence ranked the disagreements at 0.77, the preregistered bar on
its point value (the interval reaches 0.68), but not the confident confusions between agriculture and mining. The
[technical report](https://github.com/2imi9/olmoearth_inferenceX/blob/main/report/main.pdf) and the
[documentation](https://olmoearth-inferencex.readthedocs.io/en/latest/Summary/) give the results and their limits.


Development and license
-----------------------

Clone the repository, then `uv sync` and `uv run pytest`; the experiments also need `uv sync --extra encoder --extra geo`.
Apache License 2.0 ([LICENSE](https://github.com/2imi9/olmoearth_inferenceX/blob/main/LICENSE));
cite with [CITATION.cff](https://github.com/2imi9/olmoearth_inferenceX/blob/main/CITATION.cff);
questions in [GitHub issues](https://github.com/2imi9/olmoearth_inferenceX/issues).
