llama-index-readers-skim

Give your LlamaIndex pipeline the ability to read any URL — clean Markdown, ~4x smaller than raw HTML. No ads, no nav, no boilerplate.

llama-index-readers-skim is the official LlamaIndex reader for Skim — the clean reader API for AI agents. It exposes one reader, SkimReader, that turns any web page into a LlamaIndex Document of agent-ready Markdown plus structured metadata. Output is ~4x smaller than raw HTML — your pipeline ingests more pages with fewer tokens.

Two ways to pay: card plan with API key (free tier: 1,000 reads/month — skim402.com/pricing) or x402 wallet pay-per-call ($0.002 USDC on Base, no account needed).

See it before you wire it: try Skim free in your browser — 10 free skims a day, no wallet, no signup. Paste a URL, see exactly what your agent gets back.

Install

pip install llama-index-readers-skim

Quickstart (60 seconds)

1. Get a free API key

Go to skim402.com/pricing, sign up for the free plan (1,000 reads/month), and copy your sk402_... key.

2. Set the env var

export SKIM_API_KEY=sk402_your_key_here

3. Use it

from llama_index.readers.skim import SkimReader

reader = SkimReader()  # reads SKIM_API_KEY from the environment

documents = reader.load_data(urls=["https://en.wikipedia.org/wiki/HTTP_402"])
print(documents[0].text)
print(documents[0].metadata)

Each Document contains the clean Markdown as text and page metadata in metadata. ~4x smaller than raw HTML.

Alternative: pay per call with a crypto wallet

If you prefer x402 wallet pay-per-call instead of a card plan:

export SKIM_WALLET_PRIVATE_KEY=0xYOUR_BASE_WALLET_PRIVATE_KEY

Fund a dedicated Base wallet with a small USDC balance. Each read costs $0.002 on Base. Full setup guide: https://skim402.com/wallet.

Use a fresh wallet, not your personal one. This wallet's private key signs payment authorizations on your machine — treat it like a hot wallet for paying $0.002 tolls, not a savings account.

Build an index from web pages

SkimReader returns standard LlamaIndex Document objects, so it drops straight into any ingestion pipeline:

from llama_index.core import VectorStoreIndex
from llama_index.readers.skim import SkimReader

reader = SkimReader()
documents = reader.load_data(
    urls=[
        "https://example.com/article-one",
        "https://example.com/article-two",
    ]
)

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
print(query_engine.query("What do these articles have in common?"))

Output shape

load_data returns a list of Document objects. Each Document has:

• text — the cleaned article body in Markdown.
• metadata — a dict with the source URL plus the page metadata Skim extracted:

{
    "source": "https://example.com/article",
    "title": "Example article",
    "byline": "Jane Doe",
    "publishedAt": "2025-01-15",
    "lang": "en",
    "excerpt": "A short summary...",
}

Empty and None metadata values are dropped. Set include_metadata=False to keep only the source URL.

Configuration

SkimReader takes the following parameters:

# Card key (recommended)
reader = SkimReader(api_key="sk402_...")

# Or wallet
reader = SkimReader(
    private_key="0x...",
    max_price_usd=0.005,
    include_metadata=False,
)

Security

• No outbound telemetry from this package. llama-index-readers-skim only talks to skim402.com (or whatever you set as base_url). No analytics, no error reporting, no phone-home.
• Wallet lane: the private key only signs payment authorizations locally — it never leaves your machine.

Links

• Skim website — https://skim402.com
• Pricing & free key — https://skim402.com/pricing
• Wallet setup guide — https://skim402.com/wallet
• API docs — https://skim402.com/docs
• GitHub — https://github.com/JessieJanie/skim402

License

MIT