LangChain – ScraperAPI
Give your AI agent the ability to browse websites, search Google and Amazon in just two lines of code.
The langchain-scraperapi package adds three ready-to-use LangChain tools backed by the ScraperAPI service:
| Tool class | Use it to |
|---|---|
ScraperAPITool |
Grab the HTML/text/markdown of any web page |
ScraperAPIGoogleSearchTool |
Get structured Google Search SERP data |
ScraperAPIAmazonSearchTool |
Get structured Amazon product-search data |
Installation
pip install -U langchain-scraperapi
Setup
Create an account at https://www.scraperapi.com/ and get an API key, then set it as an environment variable:
import os
os.environ["SCRAPERAPI_API_KEY"] = "your-api-key"
Quick Start
ScraperAPITool — Browse any website
Scrape HTML, text, or markdown from any webpage:
from langchain_scraperapi.tools import ScraperAPITool
tool = ScraperAPITool()
# Get text content
result = tool.invoke({
"url": "https://example.com",
"output_format": "text",
"render": True
})
print(result)
Parameters:
url(required) – target page URLoutput_format–"text"|"markdown"(default returns HTML)country_code– e.g."us","de"device_type–"desktop"|"mobile"premium– use premium proxiesrender– run JavaScript before returning contentkeep_headers– include response headers
ScraperAPIGoogleSearchTool — Structured Google Search
Get structured Google Search results:
from langchain_scraperapi.tools import ScraperAPIGoogleSearchTool
google_search = ScraperAPIGoogleSearchTool()
results = google_search.invoke({
"query": "what is langchain",
"num": 20,
"output_format": "json"
})
print(results)
Parameters:
query(required) – search termsoutput_format–"json"(default) or"csv"country_code,tld,num,hl,gl– optional search modifiers
ScraperAPIAmazonSearchTool — Structured Amazon Search
Get structured Amazon product search results:
from langchain_scraperapi.tools import ScraperAPIAmazonSearchTool
amazon_search = ScraperAPIAmazonSearchTool()
products = amazon_search.invoke({
"query": "noise cancelling headphones",
"tld": "co.uk",
"page": 2
})
print(products)
Parameters:
query(required) – product search termsoutput_format–"json"(default) or"csv"country_code,tld,page– optional search modifiers
Example: AI Agent that can browse the web
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_scraperapi.tools import ScraperAPITool
# Set up tools and LLM
tools = [ScraperAPITool()]
llm = ChatOpenAI(model_name="gpt-4o", temperature=0)
# Create prompt
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant that can browse websites. Use ScraperAPITool to access web content."),
("human", "{input}"),
MessagesPlaceholder(variable_name="agent_scratchpad"),
])
# Create and run agent
agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
response = agent_executor.invoke({
"input": "Browse hackernews and summarize the top story"
})
Documentation
For complete parameter details and advanced usage, see the ScraperAPI documentation.
Metadata
Release files for langchain-scraperapi 0.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| langchain_scraperapi-0.1.2.tar.gz | 8.1 kB | Without provenance | Details for langchain_scraperapi-0.1.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| langchain_scraperapi-0.1.2-py3-none-any.whl | Python 3 | none | any | Details for langchain_scraperapi-0.1.2-py3-none-any.whl |
Total release size: 16.8 kB
Release files / langchain_scraperapi-0.1.2.tar.gz
| Download URL | langchain_scraperapi-0.1.2.tar.gz |
|---|---|
| Size | 8.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c5f1852be44bdebd3a12675ad074e4cce57cc0c46af698c6df6ddaefd37d5166
|
|
BLAKE2b-256 checksum How to use checksums |
fac2fed50cb8b6c2cb28ece5e58acca614e3d1bb10c1e8fbca1c45cfd3890ee4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.4 CPython/3.13.5 Darwin/24.6.0
|
Release files / langchain_scraperapi-0.1.2-py3-none-any.whl
| Download URL | langchain_scraperapi-0.1.2-py3-none-any.whl |
|---|---|
| Size | 8.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
33b84aeb258c5ce285d9edd6128fb8c895039649746fdfa8b3067c7595206f35
|
|
BLAKE2b-256 checksum How to use checksums |
cdf5d15a221d1b0541623b6b2d9ab15e1771113df02474a75211d064b9271184
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.4 CPython/3.13.5 Darwin/24.6.0
|
Security for langchain-scraperapi 0.1.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.