Skip to main content

CatWeb

LLM-powered classification, extraction, and summarization for web content.

Part of the CatLLM ecosystem. Thin wrapper around cat-stack that adds URL fetching and web-specific context injection.

Installation

pip install cat-web                  # pulls in cat-stack automatically
pip install "cat-web[pdf]"           # with PDF support
pip install "cat-web[agent]"         # Claude-subscription backend (model_source="claude-agent")
pip install "cat-web[codex-agent]"   # ChatGPT-subscription backend (model_source="codex-agent")

The subscription backends authenticate through your Claude or ChatGPT plan instead of a metered API key — pass model_source="claude-agent" (or "codex-agent") and leave api_key unset. model_source="claude-code" (the Claude Code CLI, if installed) needs no extra at all.

Quick Start

import catweb as cat

# Classify web pages by topic
results = cat.classify(
    categories=["News", "Opinion", "Tutorial", "Reference"],
    input_data=[
        "https://example.com/article1",
        "https://example.com/article2",
    ],
    api_key="your-api-key",
)

# Extract categories from web content
extracted = cat.extract(
    input_data=["https://example.com/page1", "https://example.com/page2"],
    description="Blog posts about technology",
    api_key="your-api-key",
)

# Summarize web pages
summaries = cat.summarize(
    input_data=["https://example.com/article1"],
    description="News articles",
    api_key="your-api-key",
)

How It Works

CatWeb accepts URLs as input, fetches the web content, strips HTML to plain text, and passes the text through cat-stack's classification/extraction/summarization pipeline. Original URLs are preserved in the output DataFrame's survey_input column.

You can also pass pre-fetched text directly — CatWeb auto-detects whether input is URLs or plain text.

API Reference

classify(categories, input_data, api_key, ...)

Classify web content into predefined categories.

Parameter Type Description
categories list Category names for classification
input_data list/Series URLs or text strings to classify
api_key str API key for the model provider
source_domain str Source domain (injected as prompt context)
content_type str Content type, e.g. "news article", "blog post"
web_metadata dict Additional key-value context for the prompt
timeout int URL fetch timeout in seconds (default 30)
**kwargs All cat-stack classify() parameters (models, creativity, batch_mode, etc.)

extract(input_data, api_key, ...)

Discover categories from web content.

Parameter Type Description
input_data list/Series URLs or text strings
api_key str API key
source_domain str Source domain context
content_type str Content type context
web_metadata dict Additional context
timeout int URL fetch timeout (default 30)
**kwargs All cat-stack extract() parameters

explore(input_data, api_key, ...)

Raw category extraction (with duplicates) for saturation analysis.

Same parameters as extract(), plus all cat-stack explore() parameters.

collapse_themes(input_data, api_key, ...)

Consolidate the raw label inventory from explore() into a smaller, deduplicated taxonomy (re-exported from the shared cat-stack engine): Jaro-Winkler dedup and embedding pre-merge, quality-controlled LLM merge passes, and an optional count-guided reduction to the top_n most common categories. This is the same consolidation extract() runs internally since cat-stack 2.5.0. See the cat-stack README for the full parameter table.

summarize(input_data, ...)

Summarize web content.

Parameter Type Description
input_data list/Series URLs or text strings
source_domain str Source domain context
content_type str Content type context
web_metadata dict Additional context
timeout int URL fetch timeout (default 30)
**kwargs All cat-stack summarize() parameters (api_key, description, models, etc.)

Web Utilities

from catweb import is_url, fetch_url_text, fetch_urls

# Check if a string is a URL
is_url("https://example.com")  # True
is_url("just text")            # False

# Fetch a single URL
text, error = fetch_url_text("https://example.com")

# Fetch multiple URLs
results = fetch_urls(["https://a.com", "https://b.com"])
# Returns: [(url, text, error), ...]

Multi-Model Ensemble

All cat-stack ensemble features work through **kwargs:

results = cat.classify(
    categories=["Positive", "Negative", "Neutral"],
    input_data=urls,
    models=[
        ("gpt-4o", "openai", "sk-..."),
        ("claude-sonnet-4-5-20250929", "anthropic", "sk-ant-..."),
    ],
    consensus_threshold="majority",
)

License

GPL-3.0-or-later

Metadata

Release files for cat-web 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cat-web 0.3.0
File Size Uploaded Provenance Actions
cat_web-0.3.0.tar.gz 7.5 kB Without provenance Details for cat_web-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for cat-web 0.3.0
File Interpreter ABI Platform Actions
cat_web-0.3.0-py3-none-any.whl Python 3 none any Details for cat_web-0.3.0-py3-none-any.whl

Total release size: 20.4 kB

Release files / cat_web-0.3.0.tar.gz

Download URL cat_web-0.3.0.tar.gz
Size 7.5 kB
Tags Source
SHA-256 checksum
How to use checksums
724a080fa92ec7c127a88d548012b8b1be6fd86566ee619801665765cdfe02cd
BLAKE2b-256 checksum
How to use checksums
14b770e1b6821eae15c63ebe1a7325e0d391db82c947ff9ef7d648ff887309df
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.14

Release files / cat_web-0.3.0-py3-none-any.whl

Download URL cat_web-0.3.0-py3-none-any.whl
Size 12.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5212e617b8b2a45f60c15ba44111403a22bd7a9ccfbbfdda93f67377e3c86551
BLAKE2b-256 checksum
How to use checksums
8bdb9bc7bf7f1c428473a81e70c6d5e8b508ffd416f9e7167eeb9261056334a1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.14

Security for cat-web 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page