olmoearth-inferencex 1.1.3

olmoearth_inferenceX is a tool for auditing Earth-observation prediction maps without labels. It ranks the windows of a map by the model's own confidence, so a reviewer knows where to look first, says why each window is flagged, and measures how two inferences of the same scene differ.

On the 24 tasks of Ai2's published embedding suite where a confidence margin is defined, the model's own confidence ranks its errors better than any control that sees no model, on every one.

Full documentation is available at https://olmoearth-inferencex.readthedocs.io/: the findings, the usage, the recipe and the evidence experiment by experiment.

Demo

pip install olmoearth-inferencex
oe-inferencex demo

It audits a real map, one tile of Dynamic World land cover, chosen by a rule fixed in advance (the median tile of 18, not the best one), and draws this:

Left: the 5% of windows to check first, found without labels. Middle: the same windows over the real errors, in red: 67% of them are wrong, against 19% of windows picked at random. Right: a random 5%. Most of the red lies outside the flagged windows because the map is 19% wrong and the review is 5%: no 5% could hold more than 26% of the errors, and these hold 17%; a 20% review finds 55%.

Your own map: oe-inferencex assess your_map.tif --out audit.

Setup

pip install olmoearth-inferencex        # add [geo] to read and write GeoTIFFs

To work on the repository, clone it, then uv sync and uv run pytest; the experiments also need uv sync --extra encoder --extra geo.

Apache License 2.0, see LICENSE; to cite, see CITATION.cff. For questions and suggestions, please open an issue on GitHub.