CatWeb
LLM-powered classification, extraction, and summarization for web content.
Part of the CatLLM ecosystem. Thin wrapper around cat-stack that adds URL fetching and web-specific context injection.
Installation
pip install cat-web # pulls in cat-stack automatically
pip install "cat-web[pdf]" # with PDF support
pip install "cat-web[agent]" # Claude-subscription backend (model_source="claude-agent")
pip install "cat-web[codex-agent]" # ChatGPT-subscription backend (model_source="codex-agent")
The subscription backends authenticate through your Claude or ChatGPT plan instead of a metered API key — pass model_source="claude-agent" (or "codex-agent") and leave api_key unset. model_source="claude-code" (the Claude Code CLI, if installed) needs no extra at all.
Quick Start
import catweb as cat
# Classify web pages by topic
results = cat.classify(
categories=["News", "Opinion", "Tutorial", "Reference"],
input_data=[
"https://example.com/article1",
"https://example.com/article2",
],
api_key="your-api-key",
)
# Extract categories from web content
extracted = cat.extract(
input_data=["https://example.com/page1", "https://example.com/page2"],
description="Blog posts about technology",
api_key="your-api-key",
)
# Summarize web pages
summaries = cat.summarize(
input_data=["https://example.com/article1"],
description="News articles",
api_key="your-api-key",
)
How It Works
CatWeb accepts URLs as input, fetches the web content, strips HTML to plain text, and passes the text through cat-stack's classification/extraction/summarization pipeline. Original URLs are preserved in the output DataFrame's survey_input column.
You can also pass pre-fetched text directly — CatWeb auto-detects whether input is URLs or plain text.
API Reference
classify(categories, input_data, api_key, ...)
Classify web content into predefined categories.
| Parameter | Type | Description |
|---|---|---|
categories |
list | Category names for classification |
input_data |
list/Series | URLs or text strings to classify |
api_key |
str | API key for the model provider |
source_domain |
str | Source domain (injected as prompt context) |
content_type |
str | Content type, e.g. "news article", "blog post" |
web_metadata |
dict | Additional key-value context for the prompt |
timeout |
int | URL fetch timeout in seconds (default 30) |
**kwargs |
All cat-stack classify() parameters (models, creativity, batch_mode, etc.) |
extract(input_data, api_key, ...)
Discover categories from web content.
| Parameter | Type | Description |
|---|---|---|
input_data |
list/Series | URLs or text strings |
api_key |
str | API key |
source_domain |
str | Source domain context |
content_type |
str | Content type context |
web_metadata |
dict | Additional context |
timeout |
int | URL fetch timeout (default 30) |
**kwargs |
All cat-stack extract() parameters |
explore(input_data, api_key, ...)
Raw category extraction (with duplicates) for saturation analysis.
Same parameters as extract(), plus all cat-stack explore() parameters.
collapse_themes(input_data, api_key, ...)
Consolidate the raw label inventory from explore() into a smaller, deduplicated taxonomy (re-exported from the shared cat-stack engine): Jaro-Winkler dedup and embedding pre-merge, quality-controlled LLM merge passes, and an optional count-guided reduction to the top_n most common categories. This is the same consolidation extract() runs internally since cat-stack 2.5.0. See the cat-stack README for the full parameter table.
summarize(input_data, ...)
Summarize web content.
| Parameter | Type | Description |
|---|---|---|
input_data |
list/Series | URLs or text strings |
source_domain |
str | Source domain context |
content_type |
str | Content type context |
web_metadata |
dict | Additional context |
timeout |
int | URL fetch timeout (default 30) |
**kwargs |
All cat-stack summarize() parameters (api_key, description, models, etc.) |
Web Utilities
from catweb import is_url, fetch_url_text, fetch_urls
# Check if a string is a URL
is_url("https://example.com") # True
is_url("just text") # False
# Fetch a single URL
text, error = fetch_url_text("https://example.com")
# Fetch multiple URLs
results = fetch_urls(["https://a.com", "https://b.com"])
# Returns: [(url, text, error), ...]
Multi-Model Ensemble
All cat-stack ensemble features work through **kwargs:
results = cat.classify(
categories=["Positive", "Negative", "Neutral"],
input_data=urls,
models=[
("gpt-4o", "openai", "sk-..."),
("claude-sonnet-4-5-20250929", "anthropic", "sk-ant-..."),
],
consensus_threshold="majority",
)
License
GPL-3.0-or-later
Metadata
Release files for cat-web 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| cat_web-0.3.0.tar.gz | 7.5 kB | Without provenance | Details for cat_web-0.3.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| cat_web-0.3.0-py3-none-any.whl | Python 3 | none | any | Details for cat_web-0.3.0-py3-none-any.whl |
Total release size: 20.4 kB
Release files / cat_web-0.3.0.tar.gz
| Download URL | cat_web-0.3.0.tar.gz |
|---|---|
| Size | 7.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
724a080fa92ec7c127a88d548012b8b1be6fd86566ee619801665765cdfe02cd
|
|
BLAKE2b-256 checksum How to use checksums |
14b770e1b6821eae15c63ebe1a7325e0d391db82c947ff9ef7d648ff887309df
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.14
|
Release files / cat_web-0.3.0-py3-none-any.whl
| Download URL | cat_web-0.3.0-py3-none-any.whl |
|---|---|
| Size | 12.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5212e617b8b2a45f60c15ba44111403a22bd7a9ccfbbfdda93f67377e3c86551
|
|
BLAKE2b-256 checksum How to use checksums |
8bdb9bc7bf7f1c428473a81e70c6d5e8b508ffd416f9e7167eeb9261056334a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.14
|
Security for cat-web 0.3.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.