A Python client for the scrapeready.com v1 API
Project description
Scrapeready Client
A Python client for making requests to scrapeready.com endpoints.
Features
-
website_to_text(url: str)
Scrapes text from a website. -
serp(q: str)
Retrieves SERP (Search Engine Results Page) information. -
ai_search(q: Optional[str], prompt: Optional[str], max_websites: int = 5)
Generates or uses a query to scrape top search results.
Installation
pip install scrapeready
Example Usage
Here’s how you can use the ScrapeReady client to interact with the API's main functionalities.
1. Initialize the Client
First, import the ScrapeReady class and initialize it with your API key.
from scrapeready import ScrapeReady
# Replace 'YOUR_API_KEY_HERE' with your actual API key
client = ScrapeReady(api_key="YOUR_API_KEY_HERE")
2. Scrape a Website to Text
Use the website_to_text method to scrape and extract text from a specified website.
try:
website_url = "https://www.example.com"
website_result = client.website_to_text(website_url)
print("Website to Text Result:")
print(website_result)
except Exception as e:
print(f"Error in website_to_text: {e}")
Output Example:
{
"message": "Page scraped successfully",
"remaining_credits": 99,
"page": {
"url": "https://www.example.com",
"raw_html": "<html>...</html>",
"cleaned_html": "<div>Example Domain</div>",
"relevant_html": "<div>Example Domain</div>",
"markdown": "# Example Domain\n\nThis domain is for use in illustrative examples in documents."
}
}
3. Retrieve SERP Information
Use the serp method to fetch Search Engine Results Page data based on a query.
try:
search_query = "openai chatgpt"
serp_result = client.serp(search_query)
print("\nSERP Result:")
print(serp_result)
except Exception as e:
print(f"Error in serp: {e}")
Output Example:
{
"message": "SERP scraped successfully",
"remaining_credits": 98,
"serp": {
"url": "https://www.google.com/search?q=openai+chatgpt",
"serp": {
"organicResults": [
{
"description": "ChatGPT is an AI developed by OpenAI...",
"name": "ChatGPT - OpenAI",
"url": "https://www.openai.com/chatgpt",
"rank": 1
}
// More results...
],
"pageNumber": 1,
"metadata": {
"displayedQuery": "openai chatgpt",
"searchedQuery": "openai chatgpt",
"totalOrganicResults": 100,
"dateDownloaded": "2025-01-30T12:34:56+00:00"
}
},
"httpResponseBody": "<html>...</html>"
}
}
4. Perform an AI-Powered Search
Use the ai_search method to generate or utilize a query to scrape top search results.
try:
prompt = "Who is the current President of the US?"
ai_search_result = client.ai_search(prompt=prompt, max_websites=3)
print("\nAI Search Result:")
print(ai_search_result)
except Exception as e:
print(f"Error in ai_search: {e}")
Output Example:
{
"message": "AI search completed successfully",
"remaining_credits": 96,
"ready-message": "https://www.whitehouse.gov/president-john-doe:\n# President John Doe\n\nJohn Doe is the 47th President of the United States.\n\nhttps://www.cnn.com/president-john-doe:\n# President John Doe\n\nJohn Doe was inaugurated on January 20, 2025.\n\nhttps://www.bbc.com/president-john-doe:\n# President John Doe\n\nJohn Doe has implemented several key policies...",
"pages": [
{
"url": "https://www.whitehouse.gov/president-john-doe",
"raw_html": "<html>...</html>",
"cleaned_html": "<div>President John Doe</div>",
"relevant_html": "<div>John Doe is the 47th President of the United States.</div>",
"markdown": "# President John Doe\n\nJohn Doe is the 47th President of the United States."
}
// More pages...
]
}
Handling Exceptions
All methods may raise exceptions in case of network issues, invalid API keys, or other errors. It's recommended to use blocks to handle these gracefully, as shown in the example usage above.
License
See the (./LICENSE) file for details.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file scrapeready-1.0.0.tar.gz.
File metadata
- Download URL: scrapeready-1.0.0.tar.gz
- Upload date:
- Size: 5.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.13.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
fb39e625046b0b63373df5a18142584564bd957b5944202b9bdbc43f93f9d82a
|
|
| MD5 |
9cbbc374199873c6cbec352d644caa5c
|
|
| BLAKE2b-256 |
ee2a0b9bbbddacf684caae2527b5a453b2272c9590f4ddc79ec410fd805a87d8
|
File details
Details for the file scrapeready-1.0.0-py3-none-any.whl.
File metadata
- Download URL: scrapeready-1.0.0-py3-none-any.whl
- Upload date:
- Size: 5.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.13.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
96e46f5722c3d48e1ab48dc51523e1f60febde2b776e5df357116613a79c5f29
|
|
| MD5 |
15f1e5d1f75b07b3acd27bf73558ec5e
|
|
| BLAKE2b-256 |
1f486d5e7f657c6cf1fee5a6e9a502f0242e0aa534bd6f904f75818263d6570c
|
