AdaptShot

A few-shot image classifier that knows when it doesn't know.

Documentation · Source · Changelog · Cite

Classify images from a handful of examples, on an ordinary CPU, and get an honest answer about how much to trust each prediction — including a conformal prediction set with a coverage guarantee, and an explicit "I don't know, ask a human" when the model is out of its depth.

result = learner.predict("leaf.jpg")

if result.uncertainty_flag:
    route_to_human_expert(result)     # the model declined to guess
else:
    act_on(result.prediction, result.calibrated_confidence)

Why this exists

In a rural clinic or a smallholder farm, three scarcities collide:

Most tools address the first two. The third is the hard one: neural networks are systematically overconfident, so a raw softmax score of 0.97 tells you very little about whether the answer is right. A tool that guesses confidently and wrongly is worse than no tool, because people act on it.

AdaptShot's position is that a model which reliably says "I don't know" is worth more than one that is a few points more accurate and silently wrong.

What AdaptShot is (and isn't)

It is a library. You construct a learner and call it. There is no inversion of control, no plugin system, no application lifecycle to inherit from. Your program stays in charge.

When to reach for something else

Being honest about this is more useful than pretending otherwise:

Reach for AdaptShot when all three are true: you have few labels, you have no GPU or no connectivity, and a confident wrong answer is expensive.

Installation

pip install adaptshot

Core dependencies are numpy, Pillow and onnxruntime — no CUDA, no GPU drivers, no multi-gigabyte download. That install can load a support set, predict, and save and reload a model. PyTorch is needed only for fine-tuning and for backbones other than the bundled one.

pip install "adaptshot[torch]"   # CA-EWC fine-tuning and custom backbones
pip install "adaptshot[faiss]"   # faster search for support sets >100 images
pip install "adaptshot[dev]"     # contributors: tests, linting, benchmarks

Measured, on a full support-set-to-prediction cycle:

The default backbone is mobilenet_v3_small, whose ONNX weights (4.0 MB) ship in the wheel. Embeddings agree with the torch path to 4e-06 (cosine 0.99999994), and the smoke benchmark returns the same accuracy through either. resnet18 is 44.8 MB and is not bundled; scripts/export_backbones.py generates it.

Not sure what your machine can do? Ask it — every figure below is measured on the machine running the check, none quoted from here:

import adaptshot
print(adaptshot.check_environment())

The limit, stated plainly: bundled ONNX backbones are frozen. Fine-tuning (correct() with CA-EWC) still requires adaptshot[torch].

This is enforced by CI on every push, not merely documented: tests/test_torch_optional.py blocks torch at the import system and then calls the API. TORCH_REQUIRED_OPERATIONS is empty, and the test fails if anything is added to it.

Quick start

Seven lines, no dataset, no GPU, no network. The photographs ship in the wheel.

from adaptshot import FewShotLearner
from adaptshot.data import sample_images

paths, labels = sample_images()                      # twelve real maize-leaf photos, four per class
learner = FewShotLearner()
learner.load_support_images(paths[:-1], labels[:-1])  # teach it eight
result = learner.predict(paths[-1])                   # ask about the ninth
print(result.prediction, f"{result.calibrated_confidence:.0%}", "-- ask a human" if result.uncertainty_flag else "")

That block is executed by CI on every push, with torch and the network both blocked, so it cannot drift (tests/test_quickstart.py). The images are PlantVillage's, CC BY-SA 3.0, with citation and checksums in src/adaptshot/data/samples/README.md.

Measured, from a clean virtualenv on a laptop CPU, one run each:

Site-packages afterwards: 176 MB. That is one machine on a fast connection; on a poor one the install is the part that grows, and it is the only part that touches the network at all.

When a human corrects a prediction, feed it back — the correction updates calibration and the replay buffer, so the next prediction is better informed:

learner.correct(image_path=paths[-1], true_label=labels[-1])

The five ideas behind it

You do not need to understand these to use the library, but they are what makes it different from a nearest-neighbour search with a confidence number bolted on.

The conformal guarantee is the core contribution. It is marginal coverage — over the whole distribution, not conditional on any particular class or subgroup. That distinction matters in deployment and is documented rather than glossed over.

📖 Start here — six tutorials for someone who has never coded, every command executed by the tests · How-to guides · How it works · The guarantee · API reference · Contributing

Feature status

Honest labels. Experimental means implemented and tested, not yet validated on real-world data at scale.

Memory: the 250 MB target is met on a core install. Measured peak resident set size for a full support-set-to-prediction cycle, by tests/test_memory_ceiling.py, which prints the figure on every run:

Installing torch alongside costs about 500 MB at import, because training/finetune.py guards its torch import at module scope — it does not crash without torch, but it does pay for it whenever torch is present. That is tracked in #36 and does not affect the core install.

utils/profiling.py provides a MemoryTracker for measuring your own workload.

Configuration

AdaptShotConfig is strictly typed and immutable, so a run is reproducible from its config alone. The most common fields:

📖 Full configuration reference — every field

Deployment

Runs anywhere a CPU and Python 3.9+ exist:

• Edge devices — single-board computers and older laptops with limited RAM
• Offline stations — fully functional without internet once backbone weights are cached
• Cloud — standard CPU instances, with no GPU line on the bill

Project status

Current release: v0.2.0. Pre-1.0 — the API may still change between minor versions, and each change is documented in the changelog.

Quality gates on every change: ruff, mypy --strict, 232 tests, and a deterministic smoke benchmark. See ROADMAP.md for what's planned.

Offline, proven

"Works offline" is tested, not promised. On every push, a CI job builds the wheel, installs it into a clean environment, then enters a Linux network namespace with no interfaces at all — proven by a canary connection that must fail — and inside it runs the README quickstart, the conference demo, the conformal and calibration suites, and the smoke benchmark, against the installed wheel rather than the source tree. Nothing is pre-fetched: the backbone and the sample photographs ship in the wheel.

Any dependency that adds a download, a telemetry call, or a version check in a patch release fails that job. The job is offline-wheel in .github/workflows/ci.yml; its runs are under Actions.

Results on real data

PlantVillage, 5-way 5-shot, 100 episodes, seed 42, mobilenet_v3_small. Every method sees the same episodes and the same embeddings. Accuracy is the mean over episodes with a 95% confidence interval — never a bare point estimate.

AdaptShot's accuracy is nearest-centroid's accuracy. Not approximately — across 500 queries the two disagree on zero of them. The top-1 prediction is a cosine nearest-centroid classifier, and the layers above it do not change which class comes out. If accuracy is all you need, the five-line version is the same five lines.

What those layers do change is what you are told alongside the prediction:

The threshold baseline is calibrated on the same held-out split, to the same target, and it misses it — 83.9% against a promised 90%. Conformal clears it with room to spare, and the price is roughly 1.9× the set size. That is the trade, stated plainly: the guarantee is real, it is not free, and if you do not need it the cheaper thing is genuinely cheaper.

Conformal over-covers here (98.1% against a 90% target), which costs set size it did not have to spend. That is a consequence of self-calibrating by leave-one-out on 25 support points. The score behind these sets is the distance ratio d_true / d_min (#86); the max-scaled softmax it replaced produced sets of 2.05 at 97.5%.

OOD flagged 1.5% ± 0.5 of this pool, which is entirely in-distribution.

Reproduce:

python scripts/fetch_plantvillage.py --out data/pv_bench --per-class 20 --preset benchmark
python -m benchmarks.run_plantvillage --seed 42

Latency and memory, measured on the same runs

Median and p95, not mean — tail latency is what makes a tool feel broken. All from results/plantvillage_5way5shot.json, and a test fails if this table drifts from it.

Peak memory for that cold-start cycle: 123 MB, in-process, on the core install. The benchmark harness itself peaks at 541 MB with 400 cached embeddings and four baselines in memory; that number describes the harness, not the library, and the artifact names them separately so they cannot be confused.

Measured on: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz, 16 cores, 31 GB RAM, Linux-7.1.10-200.fc44.x86_64-x86_64, Python 3.14.7, numpy 2.5.2, onnxruntime 1.29.0, core install.

Two honest caveats. The p95 of the support fit varied 2.7× between two runs on this machine (1.5 s and 4.0 s) while its median held within 2%; the tail is this laptop's, and the median is the number to plan around. And this laptop is faster than the hardware the project is built for. These figures are what the library costs here.

On ARM — the same measurement, on the kind of hardware the project is for

The laptop above is not the target. benchmarks/run_device.py takes the same measurement on whatever machine it runs on — torch-free, dataset-free, on the twelve bundled photographs — and CI runs it on every change on GitHub's ARM runner, after the full test suite and the ONNX export path have passed there. Both artifacts are committed (results/device_x86_64.json, results/device_aarch64.json); a test fails if this table drifts from them.

What this proves: AdaptShot runs on ARM without torch, the whole test suite passes there, a backbone exported on ARM is bit-for-bit the one in the wheel, and one cycle fits in about 100 MB. What it does not: a Neoverse-N2 is a server core, not a phone. It says nothing about the Cortex-A53 in a 2019 Android phone except that the architecture is not the obstacle. The lowest device actually measured is the one in the table; a phone-class figure still needs a phone.

Under distribution shift — where the guarantee stops holding

The conformal guarantee assumes the queries come from the same distribution as the calibration set. In the field they do not. This is what happens when the queries shift and the support does not — real PlantVillage photographs, blurred, darkened, re-compressed or downscaled with Pillow, 40 episodes, α = 0.1 (target 90%). Right-hand columns: the same queries after 10 shifted, labelled photographs are fed through correct(), the library's human-in-the-loop path.

Under shift the sets widen, and the guarantee still bends. Under strong blur, JPEG or downscale, coverage falls to 85.5% ± 3.6 (jpeg 5) against a 90% target, while the mean set size grows from 1.34 to 2.01. The set does say "less sure" — that is the nonconformity score doing its job (#86) — but not by enough: the calibration quantile was set on clean photographs and cannot know the queries have moved. That is exchangeability breaking, and no score fixes it.

The OOD flag is a partial early warning. Across the shifted cells its rate correlates 0.92 with the coverage lost — it rises as the guarantee bends — but fires on a minority of the affected queries at the worst levels.

A handful of in-situ corrections closes most of the gap. 10 labelled photographs of the shifted condition per episode, through correct(), move the worst cell from 85.5% ± 3.6 to 87.8% ± 2.4, with the sets back down to 1.48. Real help; the right move in the field.

Every cell traces to results/plantvillage_shift.json; reproduce with python -m benchmarks.run_shift --seed 42.

Full output including hardware, dataset commit and licence is written to results/plantvillage_5way5shot.json. The download is manual and pinned to a content-addressed commit; nothing in this repository fetches data silently.

Contributing

Contributions are welcome — code, documentation, testing, or a bug report from a real deployment.

git clone https://github.com/johnson2006christopher/adaptshot.git
cd adaptshot
pip install -e ".[dev,torch]"

ruff check src/ tests/
mypy src/adaptshot --strict
pytest tests/ -v
python -m benchmarks.run_benchmark --smoke-test --seed 42

All four must pass before a pull request. See CONTRIBUTING.md, SECURITY.md, and the Code of Conduct.

Citing

If AdaptShot is useful in your work:

@software{hassan_adaptshot,
  author  = {Hassan, Johnson Christopher},
  title   = {AdaptShot: Human-Aligned Few-Shot Vision Learning for
             Resource-Constrained Environments},
  url     = {https://github.com/johnson2006christopher/adaptshot},
  version = {0.3.0},
  year    = {2026}
}

GitHub also renders a Cite this repository button from CITATION.cff, in APA and BibTeX. The two are kept in step by tests/test_citation.py, which fails if the versions drift apart.

License

MIT — see LICENSE.

Johnson Christopher Hassan

Vision AI researcher and software engineer · Mbeya, Tanzania 🇹🇿

GitHub · LinkedIn · Email

Built for the places where AI has to work without a GPU, without the internet,
and without the luxury of being confidently wrong.