ScraperAPI Python SDK

Install

pip install scraperapi-sdk

Usage

from scraperapi_sdk import ScraperAPIClient

client = ScraperAPIClient("<API-KEY>")

# regular get request
content = client.get('https://amazon.com/')
# get request with premium
content = client.get('https://amazon.com/', params={'premium': True})

# post request
content = client.post('https://webhook.site/403e44ce-5835-4ce9-a648-188a51d9395d', headers={'Content-Type': 'application/x-www-form-urlencoded'}, data={'field1': 'data1'})

# put request
content = client.put('https://webhook.site/403e44ce-5835-4ce9-a648-188a51d9395d', headers={'Content-Type': 'application/json'}, data={'field1': 'data1'})

The content variable will contain the scraped page.

If you want to get the Response object instead of the content you can use make_request.

response = client.make_request(url='https://webhook.site/403e44ce-5835-4ce9-a648-188a51d9395d', headers={'Content-Type': 'application/json'}, data={'field1': 'data1'})
# response will be <Response [200]>

Exception

from scraperapi_sdk import ScraperAPIClient
from scraperapi_sdk.exceptions import ScraperAPIException

client = ScraperAPIClient(
    api_key=api_key,
)
try:
    result = client.post('https://webhook.site/403e44ce-5835-4ce9-a648-188a51d9395d', headers={'Content-Type': 'application/x-www-form-urlencoded'}, data={'field1': 'data1'})
    _ = result
except ScraperAPIException as e:
    print(e.original_exception)  # you can access the original exception via `.original_exception` property.

Structured Data Collection Methods

Amazon Endpoints

Amazon Product Page API

This method will retrieve product data from an Amazon product page and transform it into usable JSON.

result = client.amazon.product("<ASIN>")

result = client.amazon.product("<ASIN>", country="us", tld="com")

Read more in docs: Amazon Product Page API

Amazon Search API

This method will retrieve products for a specified search term from Amazon search page and transform it into usable JSON.

result = client.amazon.search("<QUERY>")
result = client.amazon.search("<QUERY>", country="us", tld="com")

Read more in docs: Amazon Search API

Amazon Offers API

This method will retrieve offers for a specified product from an Amazon offers page and transform it into usable JSON.

result = client.amazon.offers("<ASIN>")
result = client.amazon.offers("<ASIN>", country="us", tld="com")

Read more in docs: Amazon Offers API

Amazon Reviews API

This method will retrieve reviews for a specified product from an Amazon reviews page and transform it into usable JSON.

result = client.amazon.review("<ASIN>")
result = client.amazon.offers("<ASIN>", country="us", tld="com")

Read more in docs: Amazon Reviews API

Amazon Prices API

This method will retrieve product prices for the given ASINs and transform it into usable JSON.

result = client.amazon.prices(['<ASIN1>'])
result = client.amazon.prices(['<ASIN1>', '<ASIN2>'])
result = client.amazon.prices(['<ASIN1>', '<ASIN2>'], country="us", tld="com")

Read more in docs: Amazon Prices API