dompruner

English | 한국어

DOM AST middleware for LLM web pipelines — sits between raw HTML and your model, strips nav/ads/scripts via AST parsing, delivers original content token-efficient.

When an LLM agent fetches a web page, it receives tens of thousands of raw HTML tokens it doesn't need — navigation, ads, scripts, footers. dompruner strips all of that via DOM AST parsing and passes the original content, unchanged, directly to the model. No intermediate summarization model, no API key, no vector database.

> docs.python.org/3/library/asyncio-task.html  (query: "create_task")
> Raw HTML   44,315 tokens
> dompruner   1,275 tokens  (97.1% reduction, BM25 section filter applied)

Install

pip install dompruner

For JavaScript-rendered pages (optional):

pip install "dompruner[playwright]" && playwright install chromium

Usage

CLI

python -m dompruner https://docs.python.org/3/library/asyncio-task.html
python -m dompruner https://docs.python.org/3/library/asyncio-task.html "create_task"
python -m dompruner https://docs.python.org/3/library/asyncio-task.html --json

Python

import asyncio
from dompruner import run_pipeline

result = asyncio.run(run_pipeline(
    "https://docs.python.org/3/library/asyncio-task.html",
    query="create_task",
))
print(result.markdown)
print(f"{result.original_tokens:,} → {result.refined_tokens:,} tokens ({result.reduction_ratio:.1%})")
print(result.meta)  # {'title': '...', 'lang': 'en', 'description': '...', ...}

LangChain

from dompruner.langchain import DomPrunerLoader

docs = DomPrunerLoader(
    "https://fastapi.tiangolo.com/tutorial/body/",
    query="request body",
).load()
# docs[0].metadata includes: source, render_type, reduction_ratio,
#   bm25_confidence, cached, title, description, lang, author, ...

→ Full LangChain integration guide — Loader, BatchLoader, SitemapLoader, Retriever, Tool

How It Works

URL → fetch (httpx → UA rotation → Playwright) → render type detection
       └─ SSG: __NEXT_DATA__ RSC tree walk → Markdown
       └─ SSR/CSR: DOM AST pipeline
                    L1 FQN Router → L2 Heading Cluster → L3 CETD Engine
                    → BM25+ Section Filter → Compact Markdown

→ Architecture & internals

Related

  • dompruner-mcp — MCP server for Claude Code, Claude Desktop, Cursor, Windsurf. Zero install — npx -y dompruner-mcp.
  • LangChain integrations — listed as a third-party web loader.

License

MIT

Project links

Data verified by PyPI on Aug 12, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for dong7812 from gravatar.com dong7812

GitHub Statistics

Data verified by PyPI on Aug 12, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT

Requires

Python >=3.10

Provides Extra

dev playwright

Tags

dom langchain llm mcp rag token-reduction web-scraping

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dompruner-0.8.1.tar.gz (42.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dompruner-0.8.1-py3-none-any.whl (25.9 kB view details)

Uploaded Python 3

File details

Details for the file dompruner-0.8.1.tar.gz.

File metadata

  • Download URL: dompruner-0.8.1.tar.gz
  • Upload date:
  • Size: 42.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for dompruner-0.8.1.tar.gz
Algorithm Hash digest
SHA256 4550439cf28275a5df468a8e3d4973ada9e26995c845ea71e878de9055300957
MD5 dcc0867eb415bea7593d37133bd3d54e
BLAKE2b-256 4a28aaeb72a792f492fed917657ca5bba3ba07cf937b394c7ae2b1e9bdfa7b68

See more details on using hashes here.

Provenance

The following attestation bundles were made for dompruner-0.8.1.tar.gz:

Publisher: publish.yml on dong7812/dompruner-py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dompruner-0.8.1-py3-none-any.whl.

File metadata

  • Download URL: dompruner-0.8.1-py3-none-any.whl
  • Upload date:
  • Size: 25.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for dompruner-0.8.1-py3-none-any.whl
Algorithm Hash digest
SHA256 94277f9360ef4aaf0472796ecf7739dd297e741abb8160f19dda7b4659cb54da
MD5 5499d9025d65c00bcdc6a72ebb90ac46
BLAKE2b-256 94bb626bbf769db12cdd25add385656f70b612c4b6c3f0607840d77c46d295cd

See more details on using hashes here.

Provenance

The following attestation bundles were made for dompruner-0.8.1-py3-none-any.whl:

Publisher: publish.yml on dong7812/dompruner-py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page