rspns

Reduce HTTP response bodies for LLMs in their original format: HTML → HTML, JavaScript → JavaScript, JSON → JSON, XML → XML.

The output is a string containing only the reduced body. No node schema, metadata envelope, synthetic reference objects or renamed fields are added.

pip install .
# Development:
uv sync --extra dev
uv run pytest -q

Python 3.11+; synchronous, local processing. No LLM, browser or HTTP client required.

Command line

pip install rspns
python -m rspns page.html
python -m rspns script.js
python -m rspns page.html > page.summarized.html

The command prints only the summarized body to stdout, without a report or extra newline. Warnings and errors go to stderr; input errors return a nonzero exit code. The input file is read without being modified. Use a different filename when redirecting output. HTML, JS (.js, .mjs, .cjs), JSON and XML extensions select the format; other extensions use automatic content detection.

JavaScript defaults to conservative reduction. To enable the lossy library omission described below, including for scripts embedded in HTML:

python -m rspns script.js --js-mode analysis > script.summarized.js

Quick start

from rspns import Summarizer

body = '<form action="/login"><input name="user"><button>Login</button></form>'
output = Summarizer().summarize(body)
print(output)
assert output == body  # Already minimal: nothing to remove.

Output:

<form action="/login"><input name="user"><button>Login</button></form>

With presentation and ordinary text:

body = """<html><head><style>.card { color: red; }</style></head><body>
<h1>Welcome</h1><p>A long introduction that the LLM does not need.</p>
<div class="card"><form action="/login"><input name="user"><button>Login</button></form></div>
</body></html>"""
assert Summarizer().summarize(body) == '<form action="/login"><input name="user"><button>Login</button></form>'

Per-format behavior

A JSON array with 1,000 users remains an array with 1,000 users. Shortening a field changes only its string value, for example "abcdefghilmnopqrtsuvz" → "abcd...uvz" with suitably configured limits. There is no automatic sampling or global size cap.

These bodies are intended for LLM analysis. Runtime equivalence is not guaranteed: HTML presentation/text is deliberately removed and JSON/XML values may be shortened. JavaScript removal is deliberately conservative: unknown APIs, calls with side effects, animation completion dependencies, display/visibility changes, class toggles and control state are retained. Long JS literals are kept intact because they may be validation values, endpoints or other functional data. Third-party animation libraries are not guessed to be harmless. Scripts remain <script> elements containing code; they are never executed by the library, nor replaced with inert template elements.

JavaScript bundles for LLM analysis

Already minified bundles have almost no whitespace to remove. Beautifying first does not help the syntax-based reducer identify more removable code. For an explicitly lossy summary, enable ReductionConfig(javascript_mode="analysis"):

from rspns import ReductionConfig, SummaryConfig, Summarizer

config = SummaryConfig(reduction=ReductionConfig(javascript_mode="analysis"))
output = Summarizer(config).summarize(body, content_type="javascript")

This mode recognizes likely Vue, jQuery, Swiper, MobileDetect and flatpickr locale implementations inside numeric-keyed module tables using multiple syntax identifiers. It replaces each recognized factory body with a string statement identifying the omitted library and module. Module IDs, factory parameters, unknown modules and code outside the recognized factories remain. Network client libraries are not classified. Recognition is heuristic: customized library code can also be omitted. The output is for LLM analysis, not execution or a complete security review of dependencies. Unsupported bundle layouts and standalone libraries remain conservative; there is no guaranteed reduction ratio. Explicit JavaScript inclusions disable library omission, and protected_paths can protect individual factory bodies/subtrees. Diagnostics record original UTF-8 byte ranges and library names for every omission. The default javascript_mode="conservative" retains functional library code.

To measure a local bundle and save its reduced output and diagnostic report:

uv run python examples/reduce_javascript.py tests/testJS.js tests/testJS.summarized.js \
  --mode analysis --report /tmp/testJS.report.json

Keep the original response if you need to inspect discarded content later. A summary does not guarantee retention of every possible security clue. No content is fetched from external script URLs, XML DTDs/entities or XInclude references.

Detection and input

output = Summarizer().summarize(
    '{ "ok": true }',
    content_type="auto",  # default
    headers=[("Content-Type", "application/json")],
)
assert output == '{"ok":true}'

Accepts str or already decompressed bytes. Byte decoding uses BOMs, declared charsets, XML encoding declarations or UTF-8. Headers are pairs so duplicates survive. Optional url, status_code and source_id are recorded only in diagnostics.

auto treats Content-Type headers as hints and verifies the body: complete JSON first, then XML/HTML markup, then JavaScript syntax plus structural evidence. XML declarations/root namespaces distinguish XML; an HTML doctype or root identifies HTML. Namespaces inside CSS, scripts or nested SVG do not turn an HTML page into XML. Ambiguous markup fragments default to HTML with a diagnostic warning. Explicit MIME types and aliases (html, json, xml, js, javascript) select the strategy directly. MIME parameters and +json/+xml suffixes are supported.

HTML class-only layout wrappers are unwrapped unless a recognized inline DOM reference needs them. External scripts are retained but do not force every wrapper to survive: selectors constructed dynamically or hidden in external code cannot be inferred. Identifiers, control state and functional attributes remain available. Within links/buttons/labels, <small> and descriptive classes such as subtitle are removed only when another label survives; a sole label is retained. Explicit tag/selector inclusions can preserve this secondary content.

Malformed JSON/XML/JS is returned unchanged, with a warning available through summarize_result(). Filters on an unparseable body raise ParseError rather than silently failing to exclude content. Empty bodies return an empty string. Unsupported text receives local string shortening. Binary bodies raise ValueError from summarize(); summarize_result() provides their diagnostic metadata instead.

Filters and reduction settings

from rspns import ANY, FilterConfig, ReductionConfig, SummaryConfig, Summarizer

config = SummaryConfig(
    filters={
        "html": FilterConfig(exclude_categories={"buttons", "links"}),
        "json": FilterConfig(exclude_paths=[("users", ANY, "internal_notes")]),
        "xml": FilterConfig(
            exclude_xpath=["//api:debug"], namespaces={"api": "urn:api"},
        ),
        "javascript": FilterConfig(exclude_symbols={"debugLog"}),
    },
    reduction=ReductionConfig(
        string_threshold=512,
        string_prefix=320,
        string_suffix=128,
        protected_paths=[("users", ANY, "id")],
        remove_presentation=True,
    ),
)
output = Summarizer(config).summarize('<form><input name="x"><button>Go</button></form>')
assert output == '<form><input name="x"></form>'

• HTML: include_categories / exclude_categories, include_tags / exclude_tags, include_selectors / exclude_selectors (CSS).
• JSON: include_paths / exclude_paths, tuples of keys and integer array indices; ANY matches a single segment, whereas "*" is a literal key.
• XML: include_xpath / exclude_xpath with a namespace map. Select elements, comments or processing instructions, not scalar XPath results or attributes.
• JavaScript: include_categories / exclude_categories and exact include_symbols / exclude_symbols. Explicit exclusions can remove functional code; the renderer rejects combinations that would create invalid syntax.

Exclusions take precedence and remove subtrees. Inclusions retain enough ancestor context to interpret selected nodes. HTML categories include forms, controls, buttons, links, labels, scripts, styles, metadata, resources, comments, text and structure. JavaScript categories include functions, classes, variables, imports, exports, calls, assignments, strings, comments and logic. HTML inclusion of text, a particular presentation tag or selector can explicitly retain content otherwise removed by default. Explicit style inclusion can retain CSS. Embedded scripts/data use their corresponding language filters.

By default strings longer than 512 characters keep 320 leading characters, ..., and 128 trailing characters. Configure string_threshold=None to disable shortening. Keys, numbers, HTML attribute/control values and JS literals stay intact. Protected paths apply to subtrees. XML paths use child indices and "text" / "tail" segments.

Sampling is available only by explicit opt-in: sample_paths={("telemetry",): 10} keeps the first ten children at that path. The first matching configured path applies. Do not enable it for collections whose complete membership you need. XML mixed-content children include text/tails, so use it on collection containers carefully.

Optional diagnostics

result = Summarizer().summarize_result('{ "ok": true }')
assert result.content == result.to_text() == '{"ok":true}'
assert result.to_json() == '{"ok":true}'  # JSON response only
assert result.statistics["output_characters"] == len(result.content)
report = result.to_dict()  # Explicit report containing content and diagnostics

summarize_result() returns a SummaryResult with content, detected format, warnings, reductions, source metadata and statistics. to_report_json() serializes this optional report; never use it when you want only the reduced body for an LLM. to_json() is reserved for JSON responses and raises on HTML/XML/JS.

The reduction log records dropped sections and original lengths outside the output. References address parsed DOM paths, ordered JSON member positions, XML paths, or decoded UTF-8 byte offsets for JavaScript. Embedded script offsets are script-local. The caller owns source storage. A preprocessor that changes the body marks its source as transformed in the report.

Python hooks and custom strategies

Pipeline: preprocessors → detection/extraction → document processors → filters → native rendering → postprocessors → result validation.

from dataclasses import replace

def remove_prefix(response, context):
    return replace(response, body=response.body.removeprefix("PREFIX:"))

def adjust_document(document, context):
    # Node values/attributes/children are available before rendering.
    return document

def adjust_output(result, context):
    result.content = result.content.replace('"internal_name":', '"name":')
    return result

summarizer = Summarizer(SummaryConfig(
    preprocessors=[remove_prefix],
    document_processors=[adjust_document],
    postprocessors=[adjust_output],
))
assert summarizer.summarize('PREFIX:{ "internal_name": "Ada" }') == '{"name":"Ada"}'

Hooks receive (value, context) and return the same public model: ResponseInput, Document, or SummaryResult. Empty lists disable a stage; callables run in order. Failures name the stage and callable in HookError. Hooks are trusted Python code. Document hooks preserve the parsed paths/selectors; JS nodes also have syntax layout information, so prefer input/output hooks for quick JavaScript transformations.

Register a strategy with StrategyRegistry.register(name, strategy, recognizer=callable, media_types=[...]). It supplies extract(input, context) -> Document and render(document, context) -> str. Custom recognizers run before built-in detection and receive decoded text. Existing format extractors can be replaced. Raise ParseError for malformed content; programming/configuration errors propagate.

Development and packaging

uv sync --extra dev
uv run pytest -q
uv run python examples/measure.py
uv build

The measurement script compares native input/output character counts. A small functional form stays identical; an HTML page dominated by CSS/prose can shrink substantially. A JS file dominated by functional string data can remain almost unchanged. Preservation takes precedence over an arbitrary compression ratio.

This revision changes the earlier experimental API: replace summarizer.summarize(body).to_json() with summarizer.summarize(body). Use summarize_result(body) only when you also want diagnostics. Removed graph-output options deduplicate and compact_tables have no equivalent: native collections retain their entries and original shape.

Publishing on GitHub and PyPI

CI tests Python 3.11–3.14 and builds distributions. The release workflow uses PyPI Trusted Publishing; no PyPI API token is stored in the repository.

Before publishing, create the GitHub repository with main as its default branch, verify the PyPI package name is available, and configure a PyPI trusted publisher for the repository and workflow publish.yml, with GitHub environment pypi. Create/protect that environment in GitHub. Add the final repository URL under [project.urls] in pyproject.toml. If the package name changes, update it in the metadata and workflow URL and refresh uv.lock.

For a release, update the version/changelog, run uv lock, the tests and uv build, then publish the matching GitHub Release v<version>. See CONTRIBUTING.md and SECURITY.md. License: MIT.