Indeed Scraper API (Python)

A small Python client that turns Indeed job pages into clean JSON using the ScrapingBee web scraping API. You pass a search query and a location, and you get back a list of jobs with title, company, location, and link. The headless browser rendering, residential proxies, and HTML parsing are handled for you.

Indeed renders its results with JavaScript and defends against automated traffic, so a plain requests.get returns little. This Indeed scraper sends each request through ScrapingBee with JavaScript rendering and residential proxies switched on, then applies the extraction rules server-side so you never touch BeautifulSoup.

Install

pip install indeed-scraper-api

Quickstart

from indeed_scraper_api import IndeedScraper

scraper = IndeedScraper(api_key="YOUR_SCRAPINGBEE_API_KEY")

result = scraper.search("python developer", location="New York", start=0)
for job in result["jobs"]:
    print(job["title"], "-", job["company"], "-", job["location"])

Get an API key with 1,000 free credits at ScrapingBee; no card required.

What you get back

search() returns a dictionary with a jobs list, one entry per Indeed job card:

{
  "jobs": [
    {
      "title": "Python Developer",
      "company": "Acme Corp",
      "location": "New York, NY",
      "url": "/rc/clk?jk=..."
    }
  ]
}

The fields come straight from the CSS selectors documented in ScrapingBee's data extraction rules, so you can add your own fields by passing a custom extract_rules dictionary.

API reference

IndeedScraper(api_key, timeout=60) creates a client.

• search(query, location="", start=0, **options) returns structured jobs for one results page. Indeed pages in steps of 10, so start=10 is page two.
• pages(query, location="", count=3, **options) yields structured results for the first count pages.
• scrape(url, extract_rules=None, **options) fetches any Indeed URL. Returns rendered HTML, or JSON when you pass extract_rules.

Every method accepts these overrides:

Credit cost

With residential proxies and JavaScript rendering both on, each request costs 25 ScrapingBee credits (Premium tier plus JS). Drop premium_proxy for cheaper datacenter requests where a target allows it. See ScrapingBee pricing for the credit table.

Requirements

Python 3.8 or newer, the requests library (installed automatically), and a ScrapingBee API key. This client is an independent wrapper built on the public ScrapingBee API and is not affiliated with Indeed.

Links

• How to scrape Indeed (ScrapingBee guide)
• ScrapingBee Indeed scraper
• Data extraction rules

License

MIT