cat-stack

Domain-agnostic text, image, and PDF classification engine powered by LLMs.

cat-stack is the shared base package for the CatLLM ecosystem. It provides the core classification, extraction, exploration, and summarization engine that all domain-specific CatLLM packages build on.

Installation

pip install cat-stack

Optional extras:

pip install cat-stack[pdf]         # PDF support (PyMuPDF)
pip install cat-stack[embeddings]  # Embedding similarity scoring
pip install cat-stack[formatter]   # JSON formatter fallback model

Ecosystem

cat-stack is independently useful for classifying any text column. Domain-specific packages extend it with tuned prompts and workflows:

Installing cat-llm pulls in all of the above.

Quick Start

import catstack as cat

# Classify text into predefined categories
result = cat.classify(
    input_data=df["text_column"],
    categories=["Positive", "Negative", "Neutral"],
    models=[("gpt-4o", "openai", OPENAI_KEY)],
    filename="classified.csv"
)

Core API

classify()

Assign predefined categories to text, images, or PDFs. Supports single-model and multi-model ensemble classification with consensus voting.

cat.classify(
    input_data=df["text"],
    categories=["Cat A", "Cat B", "Cat C"],
    models=[("gpt-4o", "openai", key1), ("claude-sonnet-4-20250514", "anthropic", key2)],
    filename="results.csv"
)

Inline prompt tuning

Add prompt_tune=True to automatically optimize the classification prompt before the full run. A browser UI opens for you to correct a small sample, then the optimized prompt is used for all remaining items.

cat.classify(
    input_data=df["text"],
    categories=["Cat A", "Cat B", "Cat C"],
    models=[("gpt-4o", "openai", key)],
    prompt_tune=15,       # tune on 15 random items, then classify all
    tune_iterations=3,    # max attempts per category (default 3)
)

prompt_tune()

Standalone automatic prompt optimization. Iteratively refines classification prompts using user feedback — classify a sample, correct mistakes in the browser, and let the LLM generate targeted per-category instructions.

result = cat.prompt_tune(
    input_data=df["text"],
    categories=["Cat A", "Cat B", "Cat C"],
    api_key="your-key",
    sample_size=15,
    max_iterations=3,
)

# Use the optimized prompt for classification
cat.classify(
    input_data=df["text"],
    categories=["Cat A", "Cat B", "Cat C"],
    api_key="your-key",
    system_prompt=result["system_prompt"],
)

extract()

Discover categories from a corpus using LLM-driven exploration. Since v2.5.0, text consolidation runs the full explore → collapse_themes() pipeline (engine="collapse", the default): the entire raw label inventory reaches the semantic merge — Jaro-Winkler dedup, embedding pre-merge, quality-controlled LLM passes, then a count-guided reduction to at most max_categories. Pass engine="legacy" to reproduce pre-2.5 runs (single merge call over a truncated inventory). collapse_kwargs forwards options to collapse_themes(); max_workers parallelizes both extraction and consolidation.

cat.extract(
    input_data=df["text"],
    survey_question="What is this text about?",
    models=[("gpt-4o", "openai", key)],
)

explore()

Raw category extraction for saturation analysis.

cat.explore(
    input_data=df["text"],
    description="Describe the main themes",
    models=[("gpt-4o", "openai", key)],
)

collapse_themes()

Consolidate a long, redundant list of extracted category labels (e.g. the output of explore()) into a smaller, deduplicated taxonomy. Runs the semantic merge iteratively, then applies a single deterministic embedding re-merge over the whole result to collapse cross-batch lexical siblings (e.g. "tension" / "estrangement") that batched passes leave separate. Tuned to err toward over-segmentation (keeping categories) rather than over-merging.

# Basic: aggressive merge, auto-stop at the quality peak
cat.collapse_themes(
    input_data=raw_labels,            # list[str] or a frequency Series/dict
    api_key=key,
    description="Why did you move?",   # the survey question / context
    aggressive=True,
    passes="auto",
    user_model="gpt-4o",
)

# Per-step model assignment: a cheap model thins restatements,
# a stronger model does the conceptual merge (providers can differ)
cat.collapse_themes(
    input_data=raw_labels,
    api_key=key,
    description="Why did you move?",
    aggressive=True,
    passes="auto",
    unique_model="Qwen/Qwen2.5-72B-Instruct:together",
    unique_model_source="huggingface",
    unique_passes=1,
    merge_model="Qwen/Qwen3.6-35B-A3B:together",
    merge_model_source="huggingface",
    max_workers=8,
)

Parameters

summarize()

Summarize text or PDF documents, with optional multi-model ensemble.

cat.summarize(
    input_data=df["text"],
    models=[("gpt-4o", "openai", key)],
    filename="summaries.csv"
)

Supported Providers

OpenAI, Anthropic, Google (Gemini), Mistral, Perplexity, xAI (Grok), HuggingFace, Ollama (local models).

All providers use the same (model_name, provider, api_key) tuple format. Provider is auto-detected from model name if omitted.

Subscription backends (no API key). Three model_source values authenticate through a chat subscription instead of a metered key — leave api_key unset: "claude-agent" (Claude subscription via the Agent SDK; pip install "cat-stack[agent]"), "claude-code" (the Claude Code CLI, if installed — no extra needed), and "codex-agent" (ChatGPT subscription; pip install "cat-stack[codex-agent]"). Classification, extraction, exploration, and summarization all route through them, including images and PDF pages on the two agent backends. They use the agent CLI's own sign-in (separate from the Claude desktop app's): runs check it up front and, if you're signed out, open a single browser window to sign in once, then continue. See the cat-claws README.

Reproducibility note: the same nominal model can behave differently across access routes. In a seed-matched benchmark (Claude Sonnet 5, temperature 0), extraction through the Agent SDK produced more varied label phrasings than the direct API (89% vs. 74% unique raw labels), while final consolidated taxonomies were equivalent. If raw label counts matter to your analysis, record the access route alongside the model version.

Features

• Automatic prompt optimization (prompt_tune) — correct a small sample in a browser UI, and the system generates per-category instructions that improve accuracy
• Multi-model ensemble with consensus voting and agreement scores
• Batch API support for OpenAI, Anthropic, Google, Mistral, and xAI. Caveat for Google (Gemini): as of 2026-06, Google's batch scheduler routinely leaves small jobs (under a few dozen rows) in BATCH_STATE_PENDING for 30+ minutes — sometimes hours — before it starts processing. Google's published SLA is up to 24h. If your job is small and you want results back quickly, use batch_mode=False for Gemini; reserve batch_mode=True for large jobs where the 50% cost discount matters more than wall-clock latency. Other providers' batch APIs (OpenAI, Anthropic, xAI) typically complete small jobs in 1-3 minutes
• Prompt strategies: Chain-of-Thought, Chain-of-Verification, step-back prompting, few-shot examples
• Text, image, and PDF input auto-detection (PDF inputs are validated against the %PDF- magic-byte header before reaching PyMuPDF, so a webpage saved with .pdf extension surfaces a clear ValueError instead of silently classifying a blank rendered page as success)
• Embedding similarity tiebreaker for ensemble consensus ties
• Pilot test — validate classifications on a small sample before committing to the full run
• Provider-conditional HTTP timeouts — cloud providers use a tight 120 s per-request timeout (catches genuine hangs without waiting too long on transient API blips), and the Ollama provider uses a wider 600 s per-request / 1200 s cumulative budget (accommodates the long per-row tails that emerge when running 14B+ models on memory- constrained hardware like 16 GB Macs). Power users can override per client: UnifiedLLMClient(provider, key, model, request_timeout=900, max_total_wait=1800), or set a process-wide override with catstack._providers.set_session_timeouts(request_timeout=..., max_total_wait=...)

Future work / contributions welcome

The following items are tracked but not yet implemented. PRs welcome — each entry includes the scope I'd suggest if someone wants to pick it up.

• Standalone SambaNova provider. Currently SambaNova-hosted models are reachable through the HuggingFace router suffix (meta-llama/...:sambanova), but there's no direct provider="sambanova" path that talks to SambaNova's own OpenAI-compatible endpoint. Wiring it up means a new PROVIDER_CONFIG entry, the right base URL (https://api.sambanova.ai/v1), token-detection rules in detect_provider, and a smoke test against one of their cheap models (e.g. Meta-Llama-3.1-8B-Instruct).

• Consolidate HuggingFace-suffix dispatch. The strings "huggingface" and "huggingface-together" are currently hardcoded in ~30 dispatch sites across pdf_functions.py / image_functions.py / text_functions_ensemble.py / _chunked.py. Adding a new router suffix (e.g. huggingface-fireworks) means updating every one of them. The cleaner refactor is a single _is_openai_compatible(model_source) helper that matches anything starting with huggingface plus the static list (openai/perplexity/xai). Same shape as our existing _sanitize_google_schema helper. Touches a lot of sites but each edit is mechanical.

• Meta-LLM "Senate VP" tiebreaker + batch_mode support for embedding_tiebreaker. The existing embedding_tiebreaker=True resolves true 50/50 ties via centroid similarity, but only in synchronous ensemble mode. Two related extensions: (a) a meta-LLM tie-breaker that invokes a separate model on tied rows (tie_break="meta_model" with a configurable model); (b) extend the existing centroid tiebreaker to work inside batch_mode=True by running it after the batch results come back, before build_output_dataframes. The infrastructure for both is already in _tiebreaker.py; the meta-LLM variant would be a new resolver function called from resolve_ties_with_centroids.

• Schema-permafail retry short-circuit. When a model's classification permanently fails schema validation across all available retry budgets, the framework keeps spending API calls. A short-circuit that detects "this model + this input is producing the same invalid output N times in a row" and bails out early would save quota. Scope was narrowed earlier (after the HF-SMALL-MODEL fix reduced the wasted-retries surface area), so there's a real risk this stays low-value; recommend writing the detection metric first, instrumenting an actual run, and only building the short-circuit if the metric says it would have helped.

License

GPL-3.0-or-later