radfact-lite 0.1.0

Lightweight text-only RadFact metric with OpenAI and Ollama backends

radfact-lite

Lightweight, text-only RadFact with OpenAI and Ollama backends.

Parses reports into phrases, optionally filters negative findings, then scores bidirectional LLM entailment as logical precision / recall / F1.

Install

pip install radfact-lite

Usage

from radfact_lite import ModelConfig, PipelineModels, RadFactLitePipeline

cfg = ModelConfig(model="gpt-4o-mini", provider="openai")  # reads OPENAI_API_KEY
models = PipelineModels(parse_model=cfg, entailment_model=cfg, filtering_model=cfg)
pipeline = RadFactLitePipeline(models)

aggregate, per_sample = pipeline.compute_radfact(
    candidates_by_id={"id1": "No pleural effusion. Mild bibasilar atelectasis."},
    references_by_id={"id1": "Mild bibasilar atelectasis. No pneumothorax."},
    filter_negatives=True,
)
print(aggregate.logical_f1)

For Ollama: ModelConfig(model="llama3.1", provider="ollama") (default base URL http://localhost:11434/v1).

Pass pre-parsed phrase lists instead of narrative text with is_narrative_text=False.

New modalities

Add prompt files under src/prompts/{report_to_phrases,negative_filtering,entailment}/<modality>/ and a value to ReportType in src/rf_types.py.