anybrowse

Turn any URL into LLM-ready Markdown. Python SDK for anybrowse.dev.

Handles JavaScript, dynamic pages, paywalls, and anti-bot measures — so your agents don't have to.

Install

pip install anybrowse

With LangChain support:

pip install anybrowse[langchain]

With LlamaIndex support:

pip install anybrowse[llamaindex]

Everything:

pip install anybrowse[all]

Quick Start

from anybrowse import AnybrowseClient

client = AnybrowseClient()  # uses free tier
# or: client = AnybrowseClient(api_key="your-key")

result = client.scrape("https://techcrunch.com")
print(result.markdown)

results = client.search("web scraping API 2025")
for r in results:
    print(r.title, r.url)

Client Usage

from anybrowse import AnybrowseClient

client = AnybrowseClient(api_key="your-api-key")  # optional for free tier

# Scrape a page → clean Markdown
page = client.scrape("https://news.ycombinator.com")
print(page.markdown)

# Scrape with context (better extraction)
page = client.scrape(
    "https://stripe.com/pricing",
    context="Extract all pricing tiers and their features"
)

# Crawl a site (multiple pages)
pages = client.crawl("https://docs.example.com", limit=20)

# Web search → Markdown results
results = client.search("best Python web scraping libraries 2024", count=5)
for r in results:
    print(r.title, r.url)

Convenience Functions

import anybrowse

# No client needed
result = anybrowse.scrape("https://example.com")
print(result.markdown)

results = anybrowse.search("AI news today", count=5)

LangChain Integration

from anybrowse.langchain import AnybrowseScrape, AnybrowseSearch, get_tools
from langchain.agents import initialize_agent, AgentType
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model="gpt-4")
tools = get_tools(api_key="your-api-key")

agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS)
agent.run("Research the latest news on AI agents and summarize the top 3 stories")

LlamaIndex Integration

from anybrowse.llamaindex import AnybrowseReader

reader = AnybrowseReader(api_key="your-api-key")
docs = reader.load_data([
    "https://docs.python.org/3/library/asyncio.html",
    "https://fastapi.tiangolo.com/tutorial/",
])

from llama_index.core import VectorStoreIndex
index = VectorStoreIndex.from_documents(docs)
query_engine = index.as_query_engine()
response = query_engine.query("How do I use async with FastAPI?")

MCP (Model Context Protocol)

Use anybrowse directly in Claude Desktop:

{
  "mcpServers": {
    "anybrowse": {
      "command": "npx",
      "args": ["-y", "anybrowse-mcp"],
      "env": {
        "ANYBROWSE_API_KEY": "your-api-key"
      }
    }
  }
}

→ Full MCP setup guide

Free Tier

You get 10 free scrapes per day. To upgrade to 50 per day, enter your email:

from anybrowse import upgrade_free_tier
upgrade_free_tier("you@example.com")
# You now have 50 free scrapes per day

Or with curl:

curl -X POST https://anybrowse.dev/upgrade-free \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com"}'

Need more? → anybrowse.dev/pricing

Links

• 🌐 anybrowse.dev
• 📖 Docs
• 💬 GitHub