feedloop 1.5.1

The fastest way to collect human preference data for LLMs

feedloop

The fastest way to collect human preference data for LLMs.

feedloop is a free developer tool that lets you collect human feedback on LLM outputs — directly from your Python code, with zero configuration. Submit pairs of model responses, review them in a local browser UI, and export a fine-tuning dataset in minutes.

---

Features

• Zero setup — pip install feedloop and you're running
• Local-first — everything runs on your machine, no cloud account needed
• Non-blocking SDK — feedloop.compare() returns immediately, your script keeps running
• Built-in review UI — side-by-side browser interface with keyboard shortcuts
• DPO-ready export — outputs standard {"prompt", "chosen", "rejected"} JSONL
• Uncertainty filtering — skip low-uncertainty comparisons automatically, focus human attention where it matters
• Training script included — generates a ready-to-run TRL DPO fine-tuning script from your data
• Session scoping — each run is isolated; data persists across sessions in SQLite
• Model agnostic — works with OpenAI, Anthropic, Hugging Face, Ollama, or any LLM

---

Use Cases

• Model comparison — compare GPT-4o vs Claude, or two versions of your own model
• Fine-tuning data collection — build a DPO preference dataset without a labelling platform
• Evaluation loops — quickly understand where your model falls short by seeing what humans prefer
• Active learning — use uncertainty scores to only review the comparisons that matter most
• Iterative improvement — collect feedback → fine-tune → re-run → repeat

---

Installation

pip install feedloop

Requires Python 3.10+. No other dependencies or accounts needed.

---

Quick Start

import feedloop

# Start the server (opens browser automatically)
feedloop.start()

# Submit pairs of outputs for review
feedloop.compare(
    prompt="Explain recursion to a 10-year-old.",
    outputs=[
        "Recursion is when a function calls itself...",
        "Imagine you're looking for a book in a library...",
    ],
    metadata={"model_a": "gpt-4o-mini", "model_b": "gpt-4o"},
)

# Rate in the browser, then export
feedloop.export("preferences.jsonl")

A browser tab opens at http://localhost:7856. Pick the better response with a click or keyboard shortcut (1, 2, or S to skip). When you're done, export to a DPO-ready JSONL file.

---

Uncertainty-Based Filtering

Only review comparisons where your model is unsure — skip the easy ones automatically:

feedloop.start(uncertainty_threshold=0.6)

feedloop.compare(
    prompt="...",
    outputs=[response_a, response_b],
    uncertainty=0.85,  # above threshold → sent to human
)

feedloop.compare(
    prompt="...",
    outputs=[response_a, response_b],
    uncertainty=0.3,   # below threshold → auto-skipped
)

---

Exported Data Format

{"prompt": "Explain recursion...", "chosen": "Imagine you're looking for a book...", "rejected": "Recursion is when a function calls itself..."}

Compatible with TRL DPOTrainer, OpenRLHF, and any custom pipeline that accepts preference pairs.

---

Documentation

Full guide, API reference, and examples: turingspark.com/tools/feedloop