txt2dataset

A package for building, standardizing and validating datasets using language models. Supports normal API as well as batch API.

• Get a Gemini API Key

Models Supported

• Gemini - Make sure to set your "GEMINI_API_KEY" to environment.
• OpenRouter - "OPENROUTER_API_KEY"
• OpenAI - "OPENAI_API_KEY"
• Custom OpenAI Endpoint - such as via Azure.

Installation

pip install txt2dataset

Usage

Schema

from pydantic import BaseModel
from typing import Optional, List
from datetime import datetime

class SingleDividend(BaseModel):
    dividend_per_share: float
    payment_date: Optional[datetime] = None
    record_date: Optional[datetime] = None
    stock_type_specified: Optional[str] = None

class DividendExtraction(BaseModel):
    info_found: bool
    data: List[SingleDividend] = []

Entries

Entries consist of an identifier and the text to be structured.

entries = [{'id':0, 'context':
    """First Business Financial Services, Inc. (the "Company") issued a press release today 
    announcing that the Company's Board of Directors declared a quarterly dividend of $0.18 
    per share on April 30, 2021, unchanged compared to the last quarterly dividend per share. 
    The dividend is payable on May 24, 2021 to shareholders of record on May 10, 2021. 
    Also on July 12, 2020 there was a payable dividend of $0.15 per share to shareholders 
    of record on July 1st, 2020."""},

    {"id":1,"context": """XYZ Corp declared a dividend of $0.25 per share, payable June 15, 2021 
    to shareholders of record as of June 1, 2021."""}
]

Prompt

Choose a prompt such as:

prompt = "Extract ALL dividend information from this text"

Dataset Builder

Choose the requests per minute that work for your api key and model.

from txt2dataset import GeminiAPIBuilder

builder = GeminiAPIBuilder()
results, errors = builder.build(prompt=prompt, schema=DividendExtraction, model="gemini-2.5-flash-lite",
               entries=entries, rpm=4_000, tpm=4_000_000, rpm_threshold=0.75, tpm_threshold=0.75)

Result

Spot Checking

Use spotcheck() to check if results look good. Highly recommended to use a more powerful model for spot checking, and cheap model for dataset generation.

spotchecks = builder.spotcheck(schema=DividendExtraction, model="gemini-2.5-flash", entries=entries,
               results=results, sample_size=10, rpm=4_000, tpm=4_000_000, rpm_threshold=0.75, tpm_threshold=0.75)

Returns a per-field verdict for each sampled entry:

[
  {'id': 0, 'fields': [
    {'name': 'dividend_per_share', 'verdict': 'correct', 'desc': '$0.18 explicitly stated in source.'},
    {'name': 'stock_type_specified', 'verdict': 'fabricated', 'desc': "'quarterly' not stated for this dividend."}
  ]},
  {'id': 1, 'fields': [
    {'name': 'dividend_per_share', 'verdict': 'correct', 'desc': '$0.25 explicitly stated.'},
  ]}
]

Customize the spot check prompt and verdict types via config:

from txt2dataset import config

config.SET_SPOT_CHECK_PROMPT("""Here is a source document and some data extracted from it.

For each extracted row, check each field value against the source document.
Only flag a value as wrong if something is egregiously wrong — meaning
the extracted value cannot be found in or inferred from the source
text with some generosity.

null values are correct when the source does not mention that field — do not flag null as debatable or fabricated simply because the field is absent from the source.

Return JSON as a list of objects, one per extracted row, each with:
- id: the row_index of the extracted row
- fields: array of objects with:
  - name: field name
  - verdict: 'correct', 'fabricated', or 'debatable'
  - desc: brief explanation of why""")
config.SET_SPOT_CHECK_SCHEMA({
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "id": {"type": "integer"},
            "fields": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "name": {"type": "string"},
                        "verdict": {"type": "string"},
                        "desc": {"type": "string"},
                    },
                    "required": ["name", "verdict", "desc"],
                },
            },
        },
        "required": ["id", "fields"],
    },
})
config.SET_SPOT_CHECK_VERDICT_COLORS({"correct": "#e8f5e9", "fabricated": "#ffebee", "slightly_off": "#fff8e1"})

Spot Checking Visualization

Use spotcheck_visualize() for an interactive visual method.

Hotkeys: LEFT/RIGHT (or A/D) to navigate, F to copy extracted rows (JSON) to clipboard, R to reject and append to reject.json, P to copy the current ID, O to reject and append to reject_id.json (configurable).

Customize hotkeys via txt2dataset.config (e.g. from txt2dataset import config; config.SET_REJECT_KEY("X")) before calling spotcheck_visualize(). Settings persist in ~/.txt2dataset/config.json (override with TXT2DATASET_CONFIG_PATH).

SET_* accepts a string or a list (to bind multiple keys). For BACK/FORWARD, include "ArrowLeft" / "ArrowRight" in the list if you want to keep arrow navigation.

Example:

from txt2dataset import config

config.SET_BACK_KEY(["ArrowLeft", "J"])
config.SET_FORWARD_KEY(["ArrowRight", "K"])
config.SET_COPY_EXTRACTED_ROWS_KEY(["F", "C"])
config.SET_COPY_ID_KEY(["P"])
config.SET_DOWNLOAD_EXTRACTED_ROWS_KEY(["O"])
config.SET_REJECT_KEY(["R", "X"])
config.SET_REJECT_FILE("my_rejects.json")

builder.spotcheck_visualize(schema=DividendExtraction, model="gemini-2.5-flash", entries=entries,
               results=results, sample_size=10, rpm=4_000, tpm=4_000_000, rpm_threshold=0.75, tpm_threshold=0.75)

Examples

See examples.