dompruner

English | 한국어

DOM AST middleware for LLM web pipelines — sits between raw HTML and your model, strips nav/ads/scripts via AST parsing, delivers original content token-efficient.

When an LLM agent fetches a web page, it receives tens of thousands of raw HTML tokens it doesn't need — navigation, ads, scripts, footers. dompruner strips all of that via DOM AST parsing and passes the original content, unchanged, directly to the model. No intermediate summarization model, no API key, no vector database.

> docs.python.org/3/library/asyncio-task.html  (query: "create_task")
> Raw HTML   44,315 tokens
> dompruner   1,275 tokens  (97.1% reduction, BM25 section filter applied)

Install

pip install dompruner

For JavaScript-rendered pages (optional):

pip install "dompruner[playwright]" && playwright install chromium

Usage

CLI

python -m dompruner https://docs.python.org/3/library/asyncio-task.html
python -m dompruner https://docs.python.org/3/library/asyncio-task.html "create_task"
python -m dompruner https://docs.python.org/3/library/asyncio-task.html --json

Python

import asyncio
from dompruner import run_pipeline

result = asyncio.run(run_pipeline(
    "https://docs.python.org/3/library/asyncio-task.html",
    query="create_task",
))
print(result.markdown)
print(f"{result.original_tokens:,} → {result.refined_tokens:,} tokens ({result.reduction_ratio:.1%})")
print(result.meta)  # {'title': '...', 'lang': 'en', 'description': '...', ...}

LangChain

from dompruner.langchain import DomPrunerLoader

docs = DomPrunerLoader(
    "https://fastapi.tiangolo.com/tutorial/body/",
    query="request body",
).load()
# docs[0].metadata includes: source, render_type, reduction_ratio,
#   bm25_confidence, cached, title, description, lang, author, ...

→ Full LangChain integration guide — Loader, BatchLoader, SitemapLoader, Retriever, Tool

How It Works

URL → fetch (httpx → UA rotation → Playwright) → render type detection
       └─ SSG: __NEXT_DATA__ RSC tree walk → Markdown
       └─ SSR/CSR: DOM AST pipeline
                    L1 FQN Router → L2 Heading Cluster → L3 CETD Engine
                    → BM25+ Section Filter → Compact Markdown

→ Architecture & internals

Related

  • dompruner-mcp — MCP server for Claude Code, Claude Desktop, Cursor, Windsurf. Zero install — npx -y dompruner-mcp.
  • LangChain integrations — listed as a third-party web loader.

License

MIT

Project links

Data verified by PyPI on Aug 12, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.1)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for dong7812 from gravatar.com dong7812

GitHub Statistics

Data verified by PyPI on Aug 12, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT

Requires

Python >=3.10

Provides Extra

dev playwright

Tags

dom langchain llm mcp rag token-reduction web-scraping

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dompruner-0.8.0.tar.gz (42.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dompruner-0.8.0-py3-none-any.whl (25.8 kB view details)

Uploaded Python 3

File details

Details for the file dompruner-0.8.0.tar.gz.

File metadata

  • Download URL: dompruner-0.8.0.tar.gz
  • Upload date:
  • Size: 42.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for dompruner-0.8.0.tar.gz
Algorithm Hash digest
SHA256 6783ac70be0e6160d222c85f7da0cac6b88fc1dce70d9e10482f4b55b627a4ea
MD5 0e36535aec16439ac1b0a70ec6ebca91
BLAKE2b-256 8655ac82d7511620eadb735b3f5f457b5686afc6146e478d82f09e1ef9a27cdf

See more details on using hashes here.

Provenance

The following attestation bundles were made for dompruner-0.8.0.tar.gz:

Publisher: publish.yml on dong7812/dompruner-py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dompruner-0.8.0-py3-none-any.whl.

File metadata

  • Download URL: dompruner-0.8.0-py3-none-any.whl
  • Upload date:
  • Size: 25.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for dompruner-0.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 3c4a45e3d31597a196e3ccd2f9901017f11e8299b15d67bdfb822aaffda41d52
MD5 9800e3ca7093ee35e1cea1bb23289160
BLAKE2b-256 e55cfda1813f1b7d5279855e367c9d551f6863f5c53951bf52cfa50c4d93e0b4

See more details on using hashes here.

Provenance

The following attestation bundles were made for dompruner-0.8.0-py3-none-any.whl:

Publisher: publish.yml on dong7812/dompruner-py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page