article-extractor 0.4.1

Pure-Python article extraction library and HTTP service - Drop-in replacement for readability-js-server

Article Extractor

Article Extractor turns arbitrary HTML into deterministic Markdown ready for ingestion pipelines.

Problem: brittle scrapers collapse when paywalls or inline scripts mutate markup.
Why now: one fetcher abstraction keeps Playwright and httpx output identical across the CLI, FastAPI server, and Python API.
Outcome: verified tutorials, a single operations runbook, and concise reference tables keep teams unblocked in production.

Value At a Glance

• Deterministic Readability-style scoring tuned for long-form docs, blogs, and knowledge bases.
• GFM-compatible Markdown and sanitized HTML identical across the CLI, FastAPI server, and Python API.
• Runtime knobs for Playwright storage, cache sizing, proxies, diagnostics, and StatsD metrics.
• Test suite coverage above 93% plus documentation that records the exact commands and outputs.

See the Docs Home for the consolidated Tutorials, Operations, Reference, and Style sections.

Choose Your Surface

Goal	Start Here	Time	Verified Commands
Run the CLI once	CLI Fast Path	< 2 min	uv pip install article-extractor, uv run article-extractor …, head ./tmp/article-extractor-cli.md
Ship the FastAPI server in Docker	Docker Service	~5 min	docker run ghcr.io/pankaj28843/article-extractor:latest, curl http://localhost:3000/health, `curl -XPOST …
Embed the library	Python Embedding	~5 min	uv run python - <<'PY' …, asyncio.run(fetch_remote())
Tune caches, networking, diagnostics, releases	Operations Runbook	task-specific	Env vars, Docker overrides, StatsD flags, gh CLI

Install (Any Environment)

pip install article-extractor           # CLI + library
pip install article-extractor[server]   # FastAPI server extras
pip install article-extractor[all]      # Playwright, httpx, FastAPI, fake-useragent

Prefer uv? Run uv pip install article-extractor or add it to pyproject.toml via uv add article-extractor[all].

Observability & Operations

• All runtimes honor diagnostics toggles (ARTICLE_EXTRACTOR_LOG_DIAGNOSTICS, ARTICLE_EXTRACTOR_METRICS_*).
• Docker image ships Chromium + Playwright state persistence; the Operations Runbook shows how to mount storage, warm caches, and inspect the queue.
• Networking, diagnostics, StatsD, validation loops, and release automation live in a single Operations Runbook.

Documentation

The MkDocs site (Overview, Tutorials, Operations, Reference, Explanations) lives at https://pankaj28843.github.io/article-extractor/. If the site is unavailable, read the Markdown sources in docs/ including style-guide.md, operations.md, and content-inventory.md.

Contributing

We welcome pull requests paired with docs. Follow the Operations Runbook for validation and include real command output in the PR description when you update documentation.

License

MIT — see LICENSE.