Python utilities for the FOCUS datasets and challenge — Foreign Object Contextual Understanding for Surgery.

The library provides dataset loaders, preprocessing pipelines, answer-format handling, and an evaluation framework for working with the FOCUS surgical VQA datasets. It can be used independently for research on foreign-object understanding in minimally invasive surgery, and also serves as the official toolkit for the ORena SAVE FOCUS challenge at MICCAI 2026.

Challenge open for registration. Submit your results and compete on the leaderboard at orena-focus-challenge.org.

Retained foreign objects are a life-threatening and preventable surgical complication. FOCUS benchmarks vision-language models on clinically relevant VQA tasks around detecting, counting, and reasoning about foreign objects in endoscopic video.

Tracks

FOCUS offers three participation tracks, each requiring a different type of visual context:

Participants may enter any subset of tracks. Each track is evaluated independently with the same hierarchical capability taxonomy.

Installation

pip install orena-focus

Quick start

from focus import FocusDataset, DatasetSplit, Track

ds = FocusDataset("heico", DatasetSplit.TEST, Track.SEGMENT)

request, reference = ds[0]
print(request.question)        # "How many sponges are visible?"
print(reference.answer)        # "2"
print(reference.format.type)   # "number"

Data preparation

Download, preprocess, and split the dataset in one script — see examples/data_preparation.py for the full walkthrough.

from focus import download
from focus.preprocessing import VideoTimestampOverlayPreprocessor, FrameExtractorPreprocessor

download("heico")

VideoTimestampOverlayPreprocessor().process(dataset="heico")
FrameExtractorPreprocessor(stride=1).process(dataset="heico")

QA annotations are fetched automatically from HuggingFace when you construct a FocusDataset.

Inference & evaluation

See examples/inference.py for an end-to-end example with Qwen3-VL.

from focus import Evaluator, Response

responses = [Response(qID=req.qID, content=my_model(req)) for req, _ in ds]

results_df, summary_df = Evaluator().run(
    requests=ds.requests,
    references=ds.references,
    responses=responses,
)
print(summary_df)

Pre-evaluation score

The pre-evaluation phase uses a single headline number: the unweighted mean over ten buckets — the five capability groups, each scored independently for in-distribution and out-of-distribution questions (flat per-question accuracy within each bucket). Missing and timed-out answers count as incorrect. It is reported as the level="pre_evaluation", name="SCORE" row of summary_df, and can also be computed directly:

score, buckets_df = Evaluator().pre_evaluation_score(results_df)

Capability taxonomy

Five capability groups, each composed of leaf capabilities assigned to questions.

Answer formats

Datasets

FOCUS builds upon two datasets: HeiCo-FOCUS (30 videos) and LapChole-FOCUS (170 videos).

HeiCo-FOCUS

The QA annotations are publicly available on HuggingFace: orena-dkfz/heico-focus-vqa.

HeiCo-FOCUS is built on the HeiCo dataset. If you use this data, please cite the original publication:

Maier-Hein, L., et al. (2021). Heidelberg colorectal data set for surgical data science in the sensor operating room. https://doi.org/10.1038/s41597-021-00882-2

The HeiCo data is released under CC BY-NC-SA 4.0 — non-commercial use only, with attribution and share-alike conditions.

LapChole-FOCUS

The LapChole-FOCUS videos comprises unpublished laparoscopic cholecystectomy recordings. It is currently only available within the scope of the FOCUS-Challenge and will be publicly released after the Challenge has ended.

To access it, send a request on HuggingFace: orena-dkfz/lapchole-focus-vqa. To access and download the data you need to register yourself with HuggingFace.

Please find the data usage agreement at the dataset card or in the LICENSE file on HuggingFace.

License

The code is licensed under the permissive MIT license. The underlying data is licensed independently, see Dataset for the data licenses details.