Firecrawl Python SDK

The Firecrawl Python SDK is a library that lets you easily search, scrape, and interact with the web for AI agents — returning clean Markdown or structured data your agents can ship with. It provides a simple and intuitive interface for the Firecrawl API.

Installation

To install the Firecrawl Python SDK, you can use pip:

pip install firecrawl-py

Usage

Get an API key from firecrawl.dev
Set the API key as an environment variable named FIRECRAWL_API_KEY or pass it as a parameter to the Firecrawl class.

Here's an example of how to use the SDK:

from firecrawl import Firecrawl
from firecrawl.types import ScrapeOptions

firecrawl = Firecrawl(api_key="fc-YOUR_API_KEY")

# Scrape a website (v2):
data = firecrawl.scrape(
  'https://firecrawl.dev', 
  formats=['markdown', 'html']
)
print(data)

# Crawl a website (v2 waiter):
crawl_status = firecrawl.crawl(
  'https://firecrawl.dev', 
  limit=100, 
  scrape_options=ScrapeOptions(formats=['markdown', 'html'])
)
print(crawl_status)

Scraping a URL

To scrape a single URL, use the scrape method. It takes the URL as a parameter and returns a document with the requested formats.

# Scrape a website (v2):
scrape_result = firecrawl.scrape('https://firecrawl.dev', formats=['markdown', 'html'])
print(scrape_result)

Video extraction

Use the video format on supported video URLs, including YouTube and TikTok. The returned video field is a signed URL to the extracted video file.

doc = firecrawl.scrape('https://www.youtube.com/watch?v=dQw4w9WgXcQ', formats=['video'])
print(doc.video)

Product extraction

Use the product format on product pages to deterministically pull structured product data (title, price, availability, variants). It is the deterministic counterpart to the LLM-based json format.

doc = firecrawl.scrape('https://firecrawl.dev', formats=['product'])
print(doc.product)

Use the menu format on menu pages to deterministically pull structured menu data (merchant, sections, items, prices, availability). It is the deterministic counterpart to the LLM-based json format.

doc = firecrawl.scrape('https://example.com/restaurant/menu', formats=['menu'])
print(doc.menu)

Parsing uploaded files

Use parse to upload local bytes/files (html, pdf, docx, etc.) as multipart form data and return the parsed document. parse does not support change tracking or browser-only options (actions, wait_for, location, mobile, screenshot, branding, audio, video).

from firecrawl import Firecrawl
from firecrawl.v2.types import ParseOptions

firecrawl = Firecrawl(api_key="fc-YOUR_API_KEY")

doc = firecrawl.parse(
  b"<!DOCTYPE html><html><body><h1>Python Parse</h1></body></html>",
  filename="upload.html",
  content_type="text/html",
  options=ParseOptions(formats=["markdown"]),
)

print(doc.markdown)

To see which file formats parse accepts, call get_parse_formats (also available on AsyncFirecrawl). Each entry has format, kind ("document" or "image"), extensions, mime_types, and available, which is False when a format is disabled on the deployment.

for f in firecrawl.get_parse_formats():
  print(f.format, f.kind, f.extensions, f.mime_types, f.available)

Crawling a Website

To crawl a website, use the crawl method. It takes the starting URL and optional parameters as arguments. You can control depth, limits, formats, and more.

crawl_status = firecrawl.crawl(
  'https://firecrawl.dev', 
  limit=100, 
  scrape_options=ScrapeOptions(formats=['markdown', 'html']),
  poll_interval=30
)
print(crawl_status)

The result is a CrawlJob. It includes the job id and any warning from the API, so you can use the job after the crawl ends, for example with get_crawl_errors(crawl_status.id). If the crawl does not finish within timeout, the SDK raises CrawlJobTimeoutError. It is a TimeoutError subclass with a job_id attribute, so you can check or cancel the job later.

from firecrawl import CrawlJobTimeoutError

try:
  crawl_status = firecrawl.crawl('https://firecrawl.dev', limit=100, timeout=120)
except CrawlJobTimeoutError as e:
  firecrawl.cancel_crawl(e.job_id)

With AsyncFirecrawl, you can stop waiting for crawl() with Task.cancel(), asyncio.timeout or asyncio.wait_for. If the crawl already has a job id, the SDK then sends a best-effort cancel for it. If the cancellation comes before start_crawl returns the id, the SDK sends no cancel, and the crawl can keep running. After the cancel, Task.cancel() gives you asyncio.CancelledError. asyncio.timeout and asyncio.wait_for give you TimeoutError. To keep the crawl running after you stop waiting, use start_crawl and wait_crawl.

Asynchronous Crawling

Looking for async operations? Check out the Async Class section below.

To enqueue a crawl asynchronously, use start_crawl. It returns the crawl ID which you can use to check the status of the crawl job.

crawl_job = firecrawl.start_crawl(
  'https://firecrawl.dev', 
  limit=100, 
  scrape_options=ScrapeOptions(formats=['markdown', 'html']),
)
print(crawl_job)

Checking Crawl Status

To check the status of a crawl job, use the get_crawl_status method. It takes the job ID as a parameter and returns the current status of the crawl job.

crawl_status = firecrawl.get_crawl_status("<crawl_id>")
print(crawl_status)

Manual Pagination (v2)

Crawl and batch scrape status responses may include a next URL when more data is available. The SDK auto-paginates by default; to page manually, disable auto-pagination and pass the opaque next URL back to the SDK.

from firecrawl.v2.types import PaginationConfig

# Crawl: fetch one page at a time
crawl_job = firecrawl.start_crawl("https://firecrawl.dev", limit=100)
status = firecrawl.get_crawl_status(
  crawl_job.id,
  pagination_config=PaginationConfig(auto_paginate=False),
)
if status.next:
  page2 = firecrawl.get_crawl_status_page(status.next)

# Batch scrape: fetch one page at a time
batch_job = firecrawl.start_batch_scrape(["https://firecrawl.dev"])
status = firecrawl.get_batch_scrape_status(
  batch_job.id,
  pagination_config=PaginationConfig(auto_paginate=False),
)
if status.next:
  page2 = firecrawl.get_batch_scrape_status_page(status.next)

Cancelling a Crawl

To cancel an asynchronous crawl job, use the cancel_crawl method. It takes the job ID of the asynchronous crawl as a parameter and returns the cancellation status. It returns False if the crawl already completed.

cancel_crawl = firecrawl.cancel_crawl(id)
print(cancel_crawl)

Map a Website

Use map to generate a list of URLs from a website. Options let you customize the mapping process, including whether to use the sitemap or include subdomains.

# Map a website (v2):
map_result = firecrawl.map('https://firecrawl.dev')
print(map_result)

Search

Use search to search the web and optionally scrape the results in the same call.

# Search the web (v2):
results = firecrawl.search("what is retrieval augmented generation?", limit=5)
for result in results.web or []:
    print(result.url, "-", result.title)

# Scrape every result as part of the search:
results = firecrawl.search(
    "firecrawl changelog",
    limit=3,
    scrape_options={"formats": ["markdown"]},
)

Results are grouped by source: .web, .news, .images and .developer.

Use categories to narrow web search to a kind of site:

results = firecrawl.search("nanopore basecalling accuracy", categories=["research"])

categories=["research"] is a website filter, not the paper index. It restricts ordinary web search to roughly 14 academic domains (arxiv.org, pubmed.ncbi.nlm.nih.gov, nature.com, biorxiv.org, ...) and returns web page results. To search papers themselves, use search_papers below.

Developer search

Use developer_search for the dedicated developer index and its complete filter and response contract. Generic search(categories=["developer"]) returns only the first passage as a description and does not accept these filters.

evidence = firecrawl.developer_search(
    "configure retry backoff",
    repos=["firecrawl/firecrawl"],
    types=["issue", "pull_request", "readme"],
    passages=3,
    language="TypeScript",
    license="MIT",
)

for result in evidence.results:
    # Result kind is the id prefix; the API intentionally omits a type field.
    print(result.id, result.license)
    for passage in result.passages:
        print(passage.text, passage.citation_url)
print(evidence.repos)  # indexed-status echoes for requested repos

developer_search also supports sources, topic, min_stars, max_stars, archived, fork, and skills="only". Supplying both repos and sources OR-combines GitHub-backed and documentation results.

Government search

Use gov_search to search the Firecrawl Government Index: primary law and regulatory material from US federal, state, and local government sources, including statutes, regulations, codes, court opinions, and other government publications. Results come back in the ordinary web-result shape. Generic search("food labeling requirements", categories=["gov"]) returns index results inside .web; like developer, it cannot be combined with other categories.

law = firecrawl.gov_search("food labeling requirements", k=5)

for result in law.data.web:
    print(result.position, result.title, result.url)

Research / paper search

Use search_papers to search Firecrawl's research paper index: ~43M paper abstracts, roughly 90% biomedical and life sciences (PubMed, bioRxiv, medRxiv), plus arXiv for physics, mathematics and computer science.

# Search the paper index (semantic search over abstracts):
papers = firecrawl.search_papers(
    "CRISPR base editing off-target effects in primary human T cells",
    k=10,
)
for paper in papers["results"]:
    print(paper["primaryId"], "-", paper["title"])

# Inspect one paper's metadata (accepts pmid:, pmcid:, doi: or arxiv: ids):
paper = firecrawl.inspect_paper("pmid:<id>")

# Read the passages inside a paper that answer a specific question:
passages = firecrawl.read_paper(
    "pmid:<id>",
    "what was the primary endpoint and the reported hazard ratio?",
    k=4,
)

# Expand along the citation graph, re-ranked for your stated intent:
related = firecrawl.related_papers(
    "pmid:<id>",
    intent="replication attempts in larger cohorts",
    k=20,
)

search_github is deprecated. The research index GitHub endpoint stops responding after 2026-11-03. Use developer_search instead: it searches GitHub issues, pull requests and readmes plus curated documentation sources, returns matched passages, and adds filters for repo, language, license and stars. It does not carry over the scores breakdown or the resultType: "web" fallback results. See the developer index docs.

Response keys are camelCase. Unlike the rest of the SDK, the research methods return the raw JSON body as a dict — they are not parsed into typed models and not normalized to snake_case. Expect paperId, primaryId, createdDate, articleRank, poolSize, and so on.

Every method above is also available on AsyncFirecrawl with the same name.

Scrape-bound interactive browsing (v2)

Use a scrape job ID to keep interacting with the replayed browser context:

doc = firecrawl.scrape(
  "https://example.com",
  actions=[{"type": "click", "selector": "a[href='/pricing']"}],
)

scrape_job_id = doc.metadata_typed.scrape_id
if not scrape_job_id:
  raise RuntimeError("Missing scrape job id")

run = firecrawl.interact(
  scrape_job_id,
  code="print(await page.url())",
  language="python",
  timeout=60,
)
print(run.stdout)

firecrawl.stop_interaction(scrape_job_id)

{/* ### Extracting Structured Data from Websites

To extract structured data from websites, use the extract method. It takes the URLs to extract data from, a prompt, and a schema as arguments. The schema is a Pydantic model that defines the structure of the extracted data.

*/}

Crawling a Website with WebSockets

To crawl a website with WebSockets, use the crawl_url_and_watch method. It takes the starting URL and optional parameters as arguments. The params argument allows you to specify additional options for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format.

# inside an async function...
nest_asyncio.apply()

# Define event handlers
def on_document(detail):
    print("DOC", detail)

def on_error(detail):
    print("ERR", detail['error'])

def on_done(detail):
    print("DONE", detail['status'])

    # Function to start the crawl and watch process
async def start_crawl_and_watch():
    # Initiate the crawl job and get the watcher
    watcher = app.crawl_url_and_watch('firecrawl.dev', exclude_paths=['blog/*'], limit=5)

    # Add event listeners
    watcher.add_event_listener("document", on_document)
    watcher.add_event_listener("error", on_error)
    watcher.add_event_listener("done", on_done)

    # Start the watcher
    await watcher.connect()

# Run the event loop
await start_crawl_and_watch()

Error Handling

The SDK handles errors returned by the Firecrawl API and raises appropriate exceptions. If an error occurs during a request, an exception will be raised with a descriptive error message.

Async Class

For async operations, you can use the AsyncFirecrawl class. Its methods mirror the Firecrawl class, but you await them.

from firecrawl import AsyncFirecrawl

firecrawl = AsyncFirecrawl(api_key="YOUR_API_KEY")

# Async Scrape (v2)
async def example_scrape():
  scrape_result = await firecrawl.scrape(url="https://example.com")
  print(scrape_result)

# Async Parse (v2)
async def example_parse():
  parse_result = await firecrawl.parse(
    b"<!DOCTYPE html><html><body><h1>Async Parse</h1></body></html>",
    filename="upload.html",
    content_type="text/html",
  )
  print(parse_result)

# Async Crawl (v2)
async def example_crawl():
  crawl_result = await firecrawl.crawl(url="https://example.com")
  print(crawl_result)

v1 compatibility

For legacy code paths, v1 remains available under firecrawl.v1 with the original method names.

from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="YOUR_API_KEY")

# v1 methods (feature‑frozen)
doc_v1 = firecrawl.v1.scrape_url('https://firecrawl.dev', formats=['markdown', 'html'])
crawl_v1 = firecrawl.v1.crawl_url('https://firecrawl.dev', limit=100)
map_v1 = firecrawl.v1.map_url('https://firecrawl.dev')

Alexandria

With a matching API deployment, search() returns complete contracts in tools. domain_tools=True adds domain matches to that same list. Find Tools provides free, progressive catalogue lookup; Search always requires a query.

result = firecrawl.search("podcast conversations about AI agents",
                         sources=["web", "alexandria"], domain_tools=True, limit=2)
print(result.tools[0].options)

catalogue = firecrawl.find_tools(providers=["particle"], limit=2)
if catalogue.items and catalogue.items[0].get("next"):
    details = firecrawl.scrape(alexandria=catalogue.items[0]["next"])
    for item in details.alexandria:
        if item.error:
            print(f"Lookup failed: {item.error.message}")
        else:
            print(item.data)

Use scrape(alexandria={"provider": ..., "capability": ..., "options": ...}) to execute a selected tool, or pass a list of up to ten calls. Check each item's error before using data. Results and execution exceptions expose request_id; reuse it with the identical payload when retrying. Automatic retries retain it. The same methods are available with await on AsyncFirecrawl.

Metadata

Release files for firecrawl 4.50.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for firecrawl 4.50.0
File	Size	Uploaded
firecrawl-4.50.0.tar.gz	266.1 kB	Oct 8, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for firecrawl 4.50.0
File	Interpreter	ABI	Platform	Reset
firecrawl-4.50.0-py3-none-any.whl 326.8 kB Oct 8, 2026	Python 3	none	any	Details

Total release size: 592.9 kB

Release files / firecrawl-4.50.0.tar.gz

Download URL	firecrawl-4.50.0.tar.gz
Size	266.1 kB
Tags	Source
SHA-256 checksum How to use checksums	`0bf25bf87d55859e772de998f016b732e1e49d3722b0ab7614439b7baee9304a`
BLAKE2b-256 checksum How to use checksums	`f3e791b76bd6b296904dba6a5cd622fc6ff2b7535612205a42b25d58cf009686`
Upload date	Oct 8, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.14.7`

Release files / firecrawl-4.50.0-py3-none-any.whl

Download URL	firecrawl-4.50.0-py3-none-any.whl
Size	326.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`df17aac37293e07bce160b716e13bc7fd81a684ca8a19966bb3af45edcd16638`
BLAKE2b-256 checksum How to use checksums	`14f19c1c7151c024fa6d7d03e168a2f6feaf90609ff6715421c76b0880091ef0`
Upload date	Oct 8, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.14.7`