olmoearth_inferenceX

olmoearth_inferenceX checks an Earth-observation map after inference: where it is likely wrong, how wrong it is, which part you can trust, and which of two maps is more accurate. It was built to check OlmoEarth's inference outputs. It reads only the per-class scores a model writes out, never runs the model and needs no torch, so it works as well on other models' maps and on published products that ship a confidence layer.

Documentation: https://olmoearth-inferencex.readthedocs.io/

Install

pip install "olmoearth-inferencex[geo]"    # Python 3.11 to 3.13; without [geo] it reads .npy only
oe-inferencex demo                          # a real Dynamic World tile; needs no data

Usage

Every command starts with oe-inferencex. The input is the model's per-class scores before the argmax, as a GeoTIFF or a .npy array of shape (C, H, W). plan says how many labels a question needs, decide turns a result into yes, no or undetermined, and --help lists each command's options. Quick start runs each one on a test map.

Limits: without labels it says where to look, not how wrong the map is. On the one test of a deployed model at a new year and place, graded at the plots of an independent random sample that had imagery, confidence ranked the errors with an AUROC of 0.64 under a one-to-one crosswalk, below the preregistered 0.70; that run cannot say whether the year, the place, the legend or the move from expert-placed points caused it. A model run on inputs unlike its training data can be confident and wrong (--condition treats each input condition on its own). Every interval and certified zone assumes the labels are right, and labelling more and rerunning is a second test, except on a sequential sample.

Use from an agent

oe-inferencex mcp is a local MCP server; nothing is hosted. With uv installed:

claude mcp add --scope user oe-inferencex -- uvx --from "olmoearth-inferencex[geo,mcp]" oe-inferencex mcp

Use a strong model: in a pilot, Claude Sonnet kept the tools' limits and Claude Haiku often dropped them. To try it, write the demo tile into an empty folder, start the agent there and ask:

uvx --from olmoearth-inferencex oe-inferencex demo

Where should I look first in oe_inferencex_demo/sample_probabilities.npy? Use windows of 1 pixel, and grade the order against oe_inferencex_demo/sample_truth.npy.

Evidence

Tested on linear probes of sixteen encoders over the tasks of Ai2's embedding suite, and on seven published land-cover products against six reference samples, with the reference points standing in for the map. On a deployed OlmoEarth model, a re-run of Ai2's configuration for its fine-tuned AWF model, the confidence ranked the map's errors with an AUROC of 0.85 at Ai2's expert-placed validation points but 0.64 on 2017 imagery at the plots of an independent random sample that had imagery (one-to-one crosswalk), below the preregistered 0.70; the record's own fine-tune of the task on OlmoEarth v1.2 gave the same verdicts (0.81 and 0.64). On a second task, Ai2's Forest Loss Driver model graded at 2026 MapBiomas Alerta alerts in the Amazon, the confidence ranked the disagreements at 0.77, the preregistered bar on its point value (the interval reaches 0.68), but not the confident confusions between agriculture and mining. The technical report and the documentation give the results and their limits.

Development and license

Clone the repository, then uv sync and uv run pytest; the experiments also need uv sync --extra encoder --extra geo. Apache License 2.0 (LICENSE); cite with CITATION.cff; questions in GitHub issues.