scavio-haystack

Scavio integration for Haystack by deepset.

Provides ScavioWebSearch, a web search component backed by the Scavio API. It returns results as Haystack Document objects (with title and URL metadata) plus the list of source links, and mirrors the existing TavilyWebSearch / ExaWebSearch components.

Scavio is a unified search API for AI agents. Get an API key at dashboard.scavio.dev.

Scope: 1 endpoint, by design

This package is a curated subset of the Scavio API, not a wrapper around all of it. It exposes exactly 1 of Scavio's 200 endpoints:

That is the whole surface. A Haystack web search component has one job — turn a query into Documents — and only Google web search maps onto it cleanly, so this package stays at one endpoint on purpose rather than growing into a general-purpose API client.

Credits: 1 credit per run / run_async call. Nothing here touches the endpoints that cost more (YouTube transcripts are 5, most Instagram calls are 2, a LinkedIn job is 30) or the body-priced ones, whose cost is a function of the request rather than a flat number (Walmart, Threads, Kuaishou and extract). A ScavioWebSearch pipeline is flat 1 credit per search.

Reaching the rest of the API

Scavio covers 200 live endpoints: 199 across 33 platforms, plus extract, which reads any URL. The platforms are Google, YouTube, Amazon, Walmart, HEB, Kroger, eBay, Target, Home Depot, Reddit, X, TikTok, TikTok Shop, Instagram, LinkedIn, Threads, Kuaishou, Zillow, Redfin, Booking.com, Tripadvisor, Airbnb, Yelp, Indeed, Glassdoor, the Apple App Store, Google Play, SEC EDGAR, Companies House, G2, Capterra, Google Ads Transparency and the Meta Ad Library. The most recent batch added 93 endpoints across 22 platforms plus extract — 21 of those platforms are new to Scavio and Walmart was rebuilt. None of them are reachable through this package, which stays at Google web search. Two ways to use them from Haystack:

• Hosted MCP server — https://mcp.scavio.dev/mcp exposes 191 tools, authenticated with an x-api-key header. No install; a curated subset loads by default and the rest is one env var away. See the MCP docs.
• The scavio SDK directly — it is already a dependency of this package: ScavioClient(api_key=...).youtube.transcript(...), .ebay.search(...), .sec.filings(...), .meta_ads.search(...), and so on. Reading an arbitrary URL is a top-level method, not a namespace: client.extract(url, format="markdown"), never client.extract.extract(...). Wrap what you need in your own Haystack component — see the SDK reference and the full endpoint list.

Installation

pip install scavio-haystack

Usage

from haystack_integrations.components.websearch.scavio import ScavioWebSearch
from haystack.utils import Secret

web_search = ScavioWebSearch(
    api_key=Secret.from_env_var("SCAVIO_API_KEY"),  # defaults to SCAVIO_API_KEY
    top_k=5,
)

result = web_search.run(query="What is Haystack by deepset?")
documents = result["documents"]
links = result["links"]

In a pipeline

from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack_integrations.components.websearch.scavio import ScavioWebSearch

template = """
Given the following web search results, answer the question.

Results:
{% for doc in documents %}{{ doc.content }}
{% endfor %}

Question: {{ query }}
Answer:
"""

pipe = Pipeline()
pipe.add_component("search", ScavioWebSearch(top_k=5))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", OpenAIGenerator())
pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")

query = "What is Haystack by deepset?"
result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})
print(result["llm"]["replies"][0])

Async support

import asyncio
from haystack_integrations.components.websearch.scavio import ScavioWebSearch

async def main():
    web_search = ScavioWebSearch(top_k=3)
    result = await web_search.run_async(query="What is Haystack by deepset?")
    print(f"Found {len(result['documents'])} documents")

asyncio.run(main())

Parameters

• api_key: API key for Scavio. Defaults to the SCAVIO_API_KEY environment variable.
• top_k: Maximum number of results to return. Defaults to 10.
• search_params: Additional parameters for the Scavio Google (v2) search endpoint. Supported keys: country_code (mapped to gl), language (mapped to hl), page (mapped to start=(page-1)*10), device, and nfpr. Any other key is passed through verbatim (e.g. google_domain, location, safe). Can be set at init time or overridden per run.

Development

This project uses Hatch.

pip install hatch

hatch run fmt-check     # lint + format check
hatch run test:unit     # unit tests
hatch run test:all      # all tests (set SCAVIO_API_KEY for integration tests)

About Scavio

Scavio is a search API for AI agents that unifies a Google Search API, Amazon Product API, Walmart Product API, YouTube API, Reddit API, TikTok API, TikTok Shop, Instagram API, X, and LinkedIn behind a single key, returning structured JSON with no scraping or proxies. Teams evaluating a Tavily alternative or SerpAPI alternative get web search plus commerce and social data on one plan.

The same key also reaches the newer verticals — eBay sold-listing price history, Target and Home Depot, Zillow and Redfin, Booking.com, Tripadvisor and Airbnb, Yelp, Indeed and Glassdoor, the Apple App Store and Google Play, G2 and Capterra, Google Ads Transparency and the Meta Ad Library, SEC EDGAR and Companies House, Threads and Kuaishou — plus Extract, which turns any URL into HTML, Markdown or text. None of these ship as Haystack components here; use the MCP server or the SDK as described above.

New accounts get 50 one-time signup credits (no monthly refill, no credit card).

License

scavio-haystack is distributed under the terms of the Apache-2.0 license.