lyrenth (Python)

Read the web as clean AIDocuments through Lyrenth, with first-class LangChain and LlamaIndex adapters.

Each URL resolves to a stable AIDocument: cleaned Markdown plus title, description, and structure, with navigation and boilerplate stripped. Reads go through Lyrenth's cross-caller cache, and for verified domains you get the publisher's canonical version.

Install

pip install lyrenth                 # core client, zero dependencies
pip install 'lyrenth[langchain]'    # + LangChain loader
pip install 'lyrenth[llamaindex]'   # + LlamaIndex reader

Get a free API key at https://lyrenth.com/signup (2,000 reads/month, no card) and set it as LYRENTH_API_KEY.

Client

from lyrenth import Lyrenth

client = Lyrenth()  # reads LYRENTH_API_KEY from the environment
doc = client.read("https://example.com/article")

print(doc.title)
print(doc.markdown)        # cleaned, agent-ready body
print(doc.word_count)
print(doc.raw)             # full v2 envelope if you need structure/economics

# just the text:
text = client.read_markdown("https://example.com/article")

# force a fresh fetch instead of the cached version:
fresh = client.read("https://example.com/article", fresh=True)

# cap the body to a context budget:
small = client.read("https://example.com/article", max_tokens=500)

# up to 20 URLs in one call (each result isolated):
for r in client.read_batch(["https://a.com", "https://b.com"]):
    print(r.document.title if r.ok else f"{r.url}: {r.error}")

LangChain

from lyrenth.langchain import LyrenthLoader

docs = LyrenthLoader(
    ["https://example.com/a", "https://example.com/b"]
).load()
# -> list[langchain_core.documents.Document]
#    page_content = cleaned Markdown
#    metadata     = {source, title, description, word_count}

Use it anywhere a LangChain loader fits (RAG ingestion, splitters, vector stores). lazy_load() streams documents one URL at a time.

LlamaIndex

from lyrenth.llamaindex import LyrenthReader

docs = LyrenthReader().load_data(["https://example.com/a"])
# -> list[llama_index.core.Document]  (text = cleaned Markdown)

Why read through Lyrenth

• Cleaner, cheaper. One stable shape per URL, far fewer tokens than raw HTML.
• Cached across callers. Repeat URLs collapse to a minimal number of origin fetches, so it is fast and origin-friendly.
• Canonical when verified. When a site's owner has verified with Lyrenth, you get the version they authored, kept fresh by their change signal.

Config

The AIDocument format is an open contract; see https://lyrenth.com/llms-full.txt. MIT licensed.