langchain-lyrenth 0.1.0

LangChain document loader for Lyrenth: read URLs as clean AIDocuments.

langchain-lyrenth

A LangChain document loader backed by Lyrenth. Reads any URL as a clean AIDocument: cleaned Markdown plus title, description, and structure, with navigation and boilerplate stripped, through Lyrenth's cached index.

Install

pip install langchain-lyrenth

Get a free API key at https://lyrenth.com/signup and set it as LYRENTH_API_KEY.

Usage

from langchain_lyrenth import LyrenthLoader

loader = LyrenthLoader(
    ["https://example.com/a", "https://example.com/b"]
)
docs = loader.load()        # list[langchain_core.documents.Document]
# docs[0].page_content -> cleaned Markdown
# docs[0].metadata      -> {"source", "title", "description", "word_count"}

lazy_load() streams one Document per URL. Pass fresh=True to force a live re-fetch instead of the cached copy, or client= to reuse a configured lyrenth.Lyrenth.

Why

• Cleaner, cheaper input for RAG: one stable shape per URL, far fewer tokens than raw HTML.
• Cached across callers, so repeat URLs are fast and origin-friendly.

The AIDocument format is an open contract; see https://lyrenth.com/llms-full.txt. MIT licensed.