crewai-piloterr

CrewAI tools for Piloterr, a web data API.

• PiloterrGoogleSearchTool searches Google.
• PiloterrScrapeWebsiteTool reads a web page and returns its title and readable text. Piloterr fetches the page on its own infrastructure, in one of three ways:
  • crawler: plain HTTP, the default;
  • rendering: a headless browser, for JavaScript pages;
  • webunlocker: for pages behind anti-bot protection.

Installation

pip install crewai-piloterr

Get an API key at app.piloterr.com and set it in the environment:

export PILOTERR_API_KEY="your-api-key"

Usage

from crewai import Agent, Crew, Task
from crewai_piloterr import PiloterrGoogleSearchTool, PiloterrScrapeWebsiteTool

researcher = Agent(
    role="Researcher",
    goal="Find and read sources on the web",
    backstory="You search, then read the best pages before answering.",
    tools=[PiloterrGoogleSearchTool(n_results=5), PiloterrScrapeWebsiteTool(engine="rendering")],
)
task = Task(description="Summarize what piloterr.com offers.", expected_output="A short summary", agent=researcher)
Crew(agents=[researcher], tasks=[task]).kickoff()

PiloterrGoogleSearchTool

Returns a JSON list of results with title, link, snippet and position.

PiloterrScrapeWebsiteTool

Returns the page title as a heading, followed by the readable text. Scripts and styles are removed.

Both tools take api_key (default PILOTERR_API_KEY). They return API errors to the agent as text, and the key never appears in them.

Development

pip install -e ".[test]"
pytest

License

MIT