Decodo tools for CrewAI

CrewAI tools for the Decodo Web Scraping API. Give your agents live web pages and search results.

Requirements

• Python 3.12 or 3.13
• A Decodo Web Scraping API key

Installation

pip install decodo-crewai

Credentials

Copy your API key from your Web Scraping API subscription on the Decodo dashboard and set it as an environment variable:

export DECODO_API_KEY="<api_key>"

Or pass it to each tool:

DecodoWebScraperTool(api_key="<api_key>")

Usage

from crewai import Agent, Crew, Task

from decodo_crewai import DecodoSearchTool, DecodoWebScraperTool

researcher = Agent(
    role="Researcher",
    goal="Find and summarize up-to-date information from the web",
    backstory="You search the web, read the most relevant pages and report what they say.",
    tools=[DecodoSearchTool(), DecodoWebScraperTool()],
)

task = Task(
    description="Find the three most recent announcements about {topic} and summarize each one.",
    expected_output="Three bullet points, each with a one-sentence summary and the source URL.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task])
print(crew.kickoff(inputs={"topic": "solid-state batteries"}))

The tools can also be called directly:

from decodo_crewai import DecodoSearchTool, DecodoWebScraperTool

print(DecodoWebScraperTool().run(url="https://ip.decodo.com"))
print(DecodoSearchTool().run(query="mechanical keyboard", source="amazon"))

For a complete multi-agent example, see the research crew: a researcher, an analyst and a writer that turn any topic into a cited report.

Tools

DecodoWebScraperTool

Arguments the agent fills in:

Constructor options:

DecodoSearchTool

Arguments the agent fills in:

DecodoEcommerceTool

Arguments the agent fills in. Pass either product_id or query:

DecodoSERPTool

Arguments the agent fills in:

Bing results are rendered in a headless browser and can take over a minute.

Errors

When a request fails, for example because of an invalid API key, a rate limit, a page that returns 404 or a network error that keeps the request from reaching Decodo, the tools return a CrewAI ToolFailure instead of raising. The agent sees what went wrong and can retry or change course, and CrewAI records the failure according to the agent's tool_failure_policy.

License

MIT