skim-haystack

Give your Haystack pipelines the ability to read any URL — clean Markdown, ~4x smaller than raw HTML. No ads, no nav, no boilerplate.

skim-haystack is the official Haystack integration for Skim — the clean reader API for AI agents. It provides one component, SkimReader, that fetches any web page and returns it as a Haystack Document (clean Markdown in content, structured metadata in meta). Output is ~4x smaller than raw HTML — your pipeline processes more pages with fewer tokens.

Two ways to pay: card plan with API key (free tier: 1,000 reads/month — skim402.com/pricing) or x402 wallet pay-per-call ($0.002 USDC on Base, no account needed).

See it before you wire it: try Skim free in your browser — 10 free skims a day, no wallet, no signup. Paste a URL, see exactly what your agent gets back.

Install

pip install skim-haystack

Quickstart (60 seconds)

1. Get a free API key

Go to skim402.com/pricing, sign up for the free plan (1,000 reads/month), and copy your sk402_... key.

2. Set the env var

export SKIM_API_KEY=sk402_your_key_here

3. Use it

from skim_haystack import SkimReader

reader = SkimReader()  # reads SKIM_API_KEY from the environment

result = reader.run(urls="https://en.wikipedia.org/wiki/HTTP_402")
print(result["documents"][0].content)

Alternative: pay per call with a crypto wallet

If you prefer x402 wallet pay-per-call instead of a card plan:

export SKIM_WALLET_PRIVATE_KEY=0xYOUR_BASE_WALLET_PRIVATE_KEY

Fund a dedicated Base wallet with a small USDC balance. Each read costs $0.002 on Base. Full setup guide: https://skim402.com/wallet.

Use a fresh wallet, not your personal one. This wallet's private key signs payment authorizations on your machine — treat it like a hot wallet for paying $0.002 tolls, not a savings account.

Or configure explicitly with Haystack Secrets:

from haystack.utils import Secret
from skim_haystack import SkimReader

reader = SkimReader(
    private_key=Secret.from_token("0x..."),
    max_price_usd=0.005,
    include_metadata=True,
)

Use it in a pipeline

SkimReader is a standard Haystack component, so it drops straight into a Pipeline. Here it fetches a page and feeds the cleaned Markdown into a prompt:

from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from skim_haystack import SkimReader

pipe = Pipeline()
pipe.add_component("reader", SkimReader())
pipe.add_component("prompt", PromptBuilder(
    template="Summarize this article in 5 bullets:\n\n{{ documents[0].content }}"
))
pipe.add_component("llm", OpenAIGenerator(model="gpt-4o-mini"))

pipe.connect("reader.documents", "prompt.documents")
pipe.connect("prompt.prompt", "llm.prompt")

result = pipe.run({"reader": {"urls": "https://en.wikipedia.org/wiki/HTTP_402"}})
print(result["llm"]["replies"][0])

Output shape

SkimReader.run(...) returns {"documents": [Document, ...]} — one Document per URL:

• Document.content — the cleaned article body in Markdown.
• Document.meta — always includes source (the URL), plus page metadata (title, byline, publishedAt, lang, excerpt, ...) unless include_metadata=False.

Configuration

SkimReader takes the following parameters:

The component supports pipeline serialization (to_dict/from_dict). When the key comes from an environment variable (the default), it is stored as a reference to that variable name — never the raw value. An inline Secret.from_token("...") is runtime-only and will not serialize.

Security

• No outbound telemetry from this package. skim-haystack only talks to skim402.com (or whatever you set as base_url). No analytics, no error reporting, no phone-home.
• Wallet lane: the private key only signs payment authorizations locally — it never leaves your machine.

Links

• Skim website — https://skim402.com
• Pricing & free key — https://skim402.com/pricing
• Wallet setup guide — https://skim402.com/wallet
• API docs — https://skim402.com/docs
• GitHub — https://github.com/JessieJanie/skim402

License

MIT