llama-index-tools-scavio

Scavio real-time search and page-extraction tools for LlamaIndex — Google, Google News, Reddit, YouTube, Amazon, and extract (read any URL) as clean Documents, plus opt-in tools to map a site's URLs, crawl a whole site, and batch-extract a URL list, with one API key. A drop-in Tavily alternative and SerpAPI alternative for LlamaIndex RAG pipelines and agents.

Install

pip install llama-index-tools-scavio

Setup

Get a Scavio API key from the Scavio Dashboard (new accounts get 50 free signup credits, one-time, no credit card). Set SCAVIO_API_KEY or pass api_key= to the tool spec.

Usage

from llama_index.tools.scavio import ScavioToolSpec

tool_spec = ScavioToolSpec()  # reads SCAVIO_API_KEY

# Direct call — returns a list of llama_index Document objects
docs = tool_spec.search("best real-time search API for AI agents", max_results=5)
for doc in docs:
    print(doc.text, doc.metadata["url"])

# Read the page behind a result — one Document holding the whole page
page = tool_spec.extract(docs[0].metadata["url"])
print(page[0].text)

Hand the tools to an agent:

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

agent = FunctionAgent(
    tools=ScavioToolSpec().to_tool_list(),
    llm=OpenAI(model="gpt-5.5"),
    system_prompt="You are a research assistant. Use Scavio for fresh web data.",
)
response = await agent.run("What are people on Reddit saying about Tavily alternatives?")

Map, crawl and batch (opt-in)

ScavioToolSpec(enable_crawl=True) adds five tools for whole sites. They are off by default because crawl and batch bill per page, so an existing tool list is unchanged until you opt in.

Crawl and batch run as background jobs. crawl_website and batch_extract start the job and wait up to wait_seconds (default 45, max 300) for it to finish, then return the job summary (id, status, pages, credits_used, next_cursor) as the first Document and up to limit pages (default 10) after it. A job still queued or running when the wait ends keeps going; read the rest with crawl_status / batch_status and its id. That keeps an agent step bounded while a large crawl (up to 6 hours) runs on.

import json

tool_spec = ScavioToolSpec(enable_crawl=True)

urls = tool_spec.map_website("https://hono.dev/docs", limit=50)[0].text.splitlines()

docs = tool_spec.crawl_website("https://hono.dev/docs", max_pages=25, include_paths=["/docs/**"])
job = json.loads(docs[0].text)        # {"id", "status", "pages", "credits_used", "next_cursor", ...}
pages = docs[1:]                      # one Document per page, metadata url/status/depth
if job["next_cursor"]:
    more = tool_spec.crawl_status(job["id"], cursor=job["next_cursor"])

batch = tool_spec.batch_extract(["https://example.com/", "https://hono.dev/docs/getting-started/basic"])

Crawl and batch bill per successful page at extract's price for the mode (normal and advanced 1 credit, ultra 2); failed and skipped pages are free, and a job stops cleanly with stopped_reason: "insufficient_credits" if the balance runs out. Reading a job is free. Crawl and map obey the site's robots.txt and stay on the start site (subdomains optional); results are kept for 3 days. Plan limits (pages per crawl / active jobs / URLs per batch): Free 10/1/10, PAYG 100/1/50, Project 500/2/100, Bootstrap 2,000/3/500, Startup 5,000/5/1,000, Growth 10,000/10/1,000. Cancellation, webhooks and idempotency keys exist on the REST API and in the scavio SDK (client.extract.cancel_crawl(...)), but are not agent tools here.

Scope: 9 endpoints, by design

ScavioToolSpec is a curated subset of the Scavio API, not a wrapper around all of it. By default it exposes 9 of Scavio's 195 endpoints (plus the five opt-in map/crawl/batch tools above) — 8 across 4 of its 31 platforms, plus the core extract endpoint — the ones that map cleanly onto RAG Documents (every function returns List[Document]):

Platforms covered here: Google (2), YouTube (4), Reddit (1), Amazon (1), plus extract, which belongs to no platform — it reads any URL. Not covered: Walmart, TikTok, TikTok Shop, Instagram, X, LinkedIn, eBay, Target, Home Depot, Zillow, Redfin, Booking, Airbnb, Tripadvisor, Yelp, Indeed, Glassdoor, the App Store, Google Play, SEC, Companies House, G2, Capterra, Google Ads, Meta Ad Library, Threads, Kuaishou, and the other 12 Google verticals (Maps, Shopping, Flights, Hotels, Trends, AI Mode, ...). That is deliberate — this is a document-retrieval tool spec, not an API client — and it stays that way.

Reaching the rest of the API

For full coverage — all 195 endpoints across 31 platforms (Google, YouTube, Amazon, Walmart, Reddit, TikTok, TikTok Shop, Instagram, X, LinkedIn, eBay, Target, Home Depot, Zillow, Redfin, Booking, Airbnb, Tripadvisor, Yelp, Indeed, Glassdoor, App Store, Google Play, SEC, Companies House, G2, Capterra, Google Ads, Meta Ad Library, Threads, Kuaishou) — point a LlamaIndex agent at the hosted MCP server at https://mcp.scavio.dev/mcp, which exposes 191 tools with no install required:

from llama_index.tools.mcp import BasicMCPClient, McpToolSpec

scavio_mcp = McpToolSpec(
    client=BasicMCPClient(
        "https://mcp.scavio.dev/mcp",
        headers={"x-api-key": "sk_live_..."},
    )
)
tools = await scavio_mcp.to_tool_list_async()

See the MCP docs. Or call the scavio SDK directly — it is already a dependency here — and wrap whichever endpoints you need.

Scavio vs Tavily vs SerpAPI

These rows compare the APIs, not this tool spec. Rows marked below the table are the ones ScavioToolSpec itself exposes; the rest are reachable via MCP or the scavio SDK.

In ScavioToolSpec: Google SERP, Google News, Reddit, YouTube, Amazon search, and URL extraction. Everything else in the Scavio column comes from MCP or the SDK.

Migrating from Tavily

- from llama_index.tools.tavily_research import TavilyToolSpec
- tool_spec = TavilyToolSpec(api_key="tvly-...")
+ from llama_index.tools.scavio import ScavioToolSpec
+ tool_spec = ScavioToolSpec(api_key="sk_live_...")

  docs = tool_spec.search("your query")

Credits

This package is not flat 1 credit. Six of the nine tools cost 1, youtube_search costs 2, youtube_transcript costs 5 — budget for it if you are bulk-ingesting transcripts into an index — and extract is priced by the tier you ask for, not per call.

extract bills only on a successful read: a dead link, a bot wall or a timeout costs nothing. Start on normal; move to advanced when the page builds its content with JavaScript, and to ultra only for the hardest pages.

Elsewhere in the API (not exposed here): YouTube streams 3, Instagram 2-10, LinkedIn 1-10 with a job at 30, Kuaishou 1-40 per endpoint, G2 5, Capterra 2, everything else 1. New accounts get 50 one-time signup credits — no monthly refill. See scavio.dev/docs and the extract docs.

About Scavio

Scavio is a real-time search API built for AI agents — 195 endpoints across 31 platforms (Google, YouTube, Amazon, Walmart, Reddit, TikTok, Instagram, X, LinkedIn, eBay, Zillow, Booking, Indeed, the App Store, SEC filings and more), plus extract to read any URL, all returning clean JSON on one key. It is a cost-effective Tavily alternative and SerpAPI alternative with broader platform coverage. Learn more in the LlamaIndex integration docs.

Links

• Scavio: https://scavio.dev
• Docs: https://scavio.dev/docs/llamaindex
• Dashboard: https://dashboard.scavio.dev