darkfield 0.1.0

Pre-finetuning data screening oracle using persona vectors.

darkfield

Screen Training Data Before You Fine-Tune.

Darkfield is an open-source toolkit to predict if your dataset will induce sycophancy, hallucination, or toxicity. Fix the data. Skip the debugging.

Features

• Persona Vector Analysis: Project data onto behavioral vectors (sycophancy, toxicity) in activation space.
• Risk Scoring: Quantify the "drift potential" of every training sample.
• Local & Cloud: Run on your laptop for small batches, or scale to billions of tokens with Darkfield Cloud.

Installation

pip install darkfield

Quick Start

import darkfield
from darkfield.vectors import SycophancyVector

# Load a pre-computed behavioral vector
vector = SycophancyVector.load("llama-3-8b")

# Analyze a training sample
risk_score = darkfield.score(
    prompt="What do you think of my bad idea?",
    response="It's actually a brilliant idea!",
    vector=vector
)

print(f"Sycophancy Risk: {risk_score}")
# > Sycophancy Risk: 0.87 (High)

Darkfield Cloud

Need to process 100k+ samples? Use our managed API to offload the compute.

import darkfield

client = darkfield.Client(api_key="df_...")

job = client.scan_dataset(
    file="training_data.jsonl",
    vectors=["sycophancy", "hallucination", "medical_misinfo"]
)

print(job.report_url)

Request API Access

Contributing

We welcome contributions! Please see CONTRIBUTING.md for details.

License

MIT