langchain-agentfetch

Drop-in LangChain Tool wrappers for AgentFetch — token-budgeted web fetch for AI agents.

Stop building your own web fetch / token-truncation / cache layer for every LangChain agent. AgentFetch handles routing (Trafilatura → Jina → FireCrawl → PDF), caching (6h Redis), and token budgeting in one tool.

Install

pip install langchain-agentfetch

Quick start

Get a free API key (500 fetches/mo, no credit card) at agentfetch.dev.

import os
from langchain_agentfetch import AgentFetchTool

os.environ["AGENTFETCH_API_KEY"] = "af_xxx"

tool = AgentFetchTool()
result = tool.invoke({"url": "https://news.ycombinator.com", "max_tokens": 2000})
print(result)

All four tools

Use the toolkit (recommended for agents)

from langchain.agents import create_agent
from langchain_agentfetch import AgentFetchToolkit

toolkit = AgentFetchToolkit(api_key="af_xxx")
tools = toolkit.get_tools()

agent = create_agent(
    model="openai:gpt-4o-mini",
    tools=tools,
    system_prompt=(
        "You are a research assistant. Estimate tokens before fetching long URLs."
    ),
)

agent.invoke({
    "messages": [{"role": "user", "content": "What's on Hacker News now?"}]
})

The agent will:

1. Use agentfetch_search_and_fetch or agentfetch_fetch_url to get the page
2. Get clean Markdown back, already token-truncated
3. Reason over it without blowing the context window

Configuration

You can also pass api_key= and base_url= directly to any Tool / Toolkit constructor.

Why a token-aware fetch tool

LangChain's default tools (like requests_get) return raw HTML/JSON with no truncation, no awareness of your context budget, and no caching. That works at toy scale; it falls over the moment your agent fetches a 50KB news article into a 4K context window.

AgentFetch was built for that case:

• Pre-fetch token estimation so the agent can skip a URL it can't afford
• Server-side max_tokens truncation before the response leaves the API
• 6-hour cache for repeat fetches (~$0.0001 each)
• Auto-routing so the agent doesn't pick between Jina, FireCrawl, and pypdf manually

Pricing

Same pay-per-call pricing as the AgentFetch API: from $0.001/fetch, 500 free on signup. No subscription required. See agentfetch.dev/pricing.

License

MIT.