olmoearth_inferenceX

olmoearth_inferenceX is a post-inference check for Earth-observation maps: where a model's map is likely wrong, how wrong it is, which part you can trust, and which of two maps is better. It reads the scores the model wrote out; it never runs the model and needs no torch.

Documentation: https://olmoearth-inferencex.readthedocs.io/

Install

pip install "olmoearth-inferencex[geo]"    # Python 3.11 to 3.13; without [geo] it reads .npy only
oe-inferencex demo                          # a real Dynamic World tile; needs no data

Four questions

Every command starts with oe-inferencex, and --help lists its options. decide turns a result into a typed answer: decide to_label_estimate.json --ask error_rate_below=0.1 prints yes, no or undetermined. Quick start runs each one on a test map and shows what it prints.

Input: the model's per-class scores, taken before the argmax, as a GeoTIFF or .npy array of shape (C, H, W): probabilities between 0 and 1, or logits with --logits. A published product's class map with its confidence band works with --confidence. A class map alone cannot be ranked. The package works on windows of 4 x 4 pixels (--patch).

Labels: for each window sample drew, write 1 or 0 in wrong (or the class you see in reference_class), and ? where a window cannot be judged.

Before you trust it

1. Without labels it says where to look, not how wrong the map is. A review set is not a sample, and estimate and certify refuse it.
2. A model run on inputs it was not trained on can be sure and wrong, and those errors are checked last. On PASTIS, a probe trained on radar plus optical and run on radar alone was 74% wrong, and 60% of its errors were as confident as a typical correct window. Give --condition a raster of each pixel's input condition, such as a cloud flag, to treat each condition on its own.
3. Every interval assumes the labels are right. For reviewer error rates you can state, estimate --reviewer-false-alarm and --reviewer-miss widen it.
4. certify can return nothing, and then says why.

Use from an agent

oe-inferencex mcp is a local MCP server; nothing is hosted. With uv installed, one line connects it to Claude Code:

claude mcp add --scope user oe-inferencex -- uvx --from "olmoearth-inferencex[geo,mcp]" oe-inferencex mcp

Use a strong model. In a pilot, Claude Sonnet kept the tools' numbers and their limits; Claude Haiku often dropped the limits. To try it, write the demo tile into an empty folder, start the agent there and ask the question below:

uvx --from olmoearth-inferencex oe-inferencex demo

Where should I look first in oe_inferencex_demo/sample_probabilities.npy? Use windows of 1 pixel, and grade the order against oe_inferencex_demo/sample_truth.npy.

Usage gives the configuration for other agents and more questions.

What has been measured

• On the 24 tasks of Ai2's embedding suite, read through linear probes on OlmoEarth Base, a review of the least confident 10% found a median 0.214 of the errors, about twice a random 10%.

• On sixteen encoders under ten probe seeds, confidence ranked the errors better than a random order in all 3,560 cells, though the lowest AUROC, 0.503, is close to chance.

• It also beat informative baselines on ground survey labels (LUCAS), Dynamic World's expert tiles and farmers' crop declarations (EuroCrops).

• Over 2,514 pairs of probe maps, 100 labels drawn where two maps differ named the more accurate one on 60% of draws, against 13% for 100 labels from the whole map (exp90, not preregistered).

• On two published land-cover products (LCMAP and Esri, 2018) graded against LCMAP's 25,000 random reference plots, the error-rate interval held its coverage, and labels drawn where the maps differ never named the wrong one; at 50 to 200 labels they named LCMAP on 6% to 29% of draws (exp92, preregistered).

• On LCMAP's own confidence layer, certified zones met their guarantee in every graded cell. At 300 labels and an allowed error of 8.9%, the default rule returned no zone on 87% of draws: the first zone it tests, a tenth of the plots, is 6% wrong and holds about 30 labels (exp93, preregistered).

• On five more reference samples, in the United States, East Africa, Europe and worldwide, and seven published products from 2001 to 2020, the intervals held their coverage, labels drawn where two products differ named the worse-agreeing one on at most 0.35% of draws, and three products' own confidences ranked their errors (exp94, preregistered).

Most of this evidence is linear probes on frozen embeddings; on published products, seven products against six reference samples have been graded, with the points standing in for the map. The technical report and the documentation give the full record and its limits.

Development and license

To work on the repository, clone it, then run uv sync and uv run pytest. The experiments also need uv sync --extra encoder --extra geo.

• License: Apache License 2.0 (LICENSE).
• Citation: CITATION.cff.
• Questions and suggestions: GitHub issues.