search-parser
Parse Google, Bing, DuckDuckGo, and Google Maps search results into typed Python objects, JSON, Markdown, or dict output.
search-parser takes raw HTML from Google, Bing, and DuckDuckGo — desktop, mobile, or Google's stripped no-JS layout — and extracts every result type — organic results, featured snippets, AI Overviews, People Also Ask, sponsored ads, shopping ads, and more — into clean, typed Python objects. It auto-detects the search engine from the HTML, so you never have to specify which parser to use.
Quick Start
from search_parser import SearchParser
parser = SearchParser()
html = open("google_results.html").read()
# JSON string (default)
json_output = parser.parse(html)
# Markdown string — great for feeding to an LLM
md_output = parser.parse(html, output_format="markdown")
# Python dict — for programmatic access
data = parser.parse(html, output_format="dict")
# Organic results are in data["results"]
for result in data["results"]:
print(f"{result['position']}. {result['title']}")
print(f" {result['url']}")
# Google rich fields are optional and live under metadata
print(f" {result['metadata'].get('rating', 'no rating')}")
# Every other result type has its own dedicated key
if data["featured_snippet"]:
print("Featured:", data["featured_snippet"]["title"])
if data["ai_overview"]:
print("AI Overview:", data["ai_overview"]["description"][:100])
for question in data["people_also_ask"]:
print("PAA:", question["title"])
# Parts without a typed field (videos, answers, knowledge panel, ads, ...) are under metadata["extras"]
extras = data["metadata"].get("extras", {})
print(extras.get("total"), len(extras.get("videos", [])))
Installation
With uv (recommended):
uv add search-parser
With pip:
pip install search-parser
Structured Google Maps results
GoogleMapsParser handles the XSSI-prefixed JSON from Google's non-JavaScript
tbm=map transport. The request builder and network access remain outside this
parsing library.
from search_parser import GoogleMapsParser
places = GoogleMapsParser().parse(response_text, query="coffee chicago")
for place in places.places:
print(place.name, place.website, place.rating, place.latitude, place.longitude)
# JSON suitable for an API response or fixture
print(places.to_json())
The parser accepts XSSI-prefixed or plain JSON, validates the positional place
record signature, suppresses duplicate Google data IDs, and decodes Google
website redirects. Invalid JSON and incomplete XSSI responses raise
GoogleMapsParseError. See the Google Maps guide for the
complete field contract and operational caveats.
Supported Result Types
| Result Type | Field | Bing | DuckDuckGo | |
|---|---|---|---|---|
| Organic results | results |
✓ | ✓ | ✓ |
| Featured snippet | featured_snippet |
✓ | ✓ | — |
| Sponsored / ads | sponsored |
✓ | — | — |
| AI Overview | ai_overview |
✓ | — | — |
| People Also Ask | people_also_ask |
✓ | — | — |
| What People Are Saying | people_saying |
✓ | — | — |
| People Also Search For | people_also_search |
✓ | — | — |
| Related Products & Services | related_products |
✓ | — | — |
| Jobs | jobs |
✓ | — | — |
| Discussions and forums | discussions |
✓ | — | — |
| Shopping ads | shopping_ads |
✓ | — | — |
| News tab articles | news |
✓ | — | — |
| Local business pack | local_businesses |
✓ | — | — |
Working with Results
SearchParser.parse() with output_format="dict" returns the full SearchResults structure:
data = parser.parse(html, output_format="dict")
# Always a list (organic results only)
for r in data["results"]:
print(r["title"], r["url"], r["description"])
print(r["metadata"].get("display_url"))
print(r["metadata"].get("rating"), r["metadata"].get("reviews"))
for sitelink in r["metadata"].get("sitelinks", []):
print(" Sitelink:", sitelink["title"], sitelink["url"])
# None or a single object
if data["featured_snippet"]:
print(data["featured_snippet"]["title"])
# None or a single object with description + sources list
if data["ai_overview"]:
overview = data["ai_overview"]
print(overview["description"])
for source in overview["metadata"]["sources"]:
print(f" - {source.get('source', source['title'])}: {source['url']}")
for detail in overview["metadata"].get("details", []):
print(f" - {detail}")
# Always a list (empty when not present)
for q in data["people_also_ask"]:
print(q["title"])
for post in data["people_saying"]:
print(post["title"], post["url"])
for item in data["people_also_search"]:
print(item["title"])
for ad in data["sponsored"]:
print(ad["title"], ad["url"])
print(ad["metadata"].get("display_url"), ad["metadata"].get("phone"))
for sitelink in ad["metadata"].get("sitelinks", []):
print(" Sitelink:", sitelink["title"])
for product in data["related_products"]:
print(product["title"])
# Jobs (title, metadata["company"], metadata["location"])
for job in data["jobs"]:
print(job["title"], job["metadata"]["company"], job["metadata"]["location"])
# Discussions (title, url, description, metadata["source"])
for disc in data["discussions"]:
print(disc["title"], disc["url"])
print(disc["metadata"]["source"])
# Shopping ads (title, metadata["price"], metadata["merchant"])
for ad in data["shopping_ads"]:
print(ad["title"], ad["metadata"]["price"], ad["metadata"]["merchant"])
# News tab articles (Google News tab, tbm=nws) — metadata["source"] and metadata["published_time"]
for article in data["news"]:
print(article["title"], article["url"])
print(article["metadata"]["source"], article["metadata"]["published_time"])
# Local business pack — every metadata key is optional. A no-JS places-only page has
# an empty data["results"] and its businesses here.
for biz in data["local_businesses"]:
meta = biz["metadata"]
print(biz["title"], meta.get("rating"), meta.get("reviews"), meta.get("category"))
print(meta.get("location"), meta.get("hours"), meta.get("price"), meta.get("phone"))
print(meta.get("details", [])) # no-JS pages: every line of the card
# Metadata
print(data["search_engine"]) # "google"
print(data["query"]) # "python web scraping"
print(data["total_results"]) # 26200000 or None
print(data["detection_confidence"]) # 0.95
print(data["metadata"].get("location"))
print(data["metadata"].get("location_source"))
print(data["metadata"].get("pagination", {}).get("next_url"))
Using the model directly
When you need the typed SearchResults object instead of a dict, call the engine parser directly. The model exposes to_json() and to_markdown() convenience methods:
from search_parser.parsers.google import GoogleParser
parser = GoogleParser()
results = parser.parse(html) # returns SearchResults
# Typed access — no dict key lookups
print(results.query)
print(results.total_results)
print(len(results.results)) # organic count
if results.featured_snippet:
print(results.featured_snippet.title)
if results.ai_overview:
print(results.ai_overview.description)
sources = results.ai_overview.metadata["sources"]
details = results.ai_overview.metadata.get("details", [])
for result in results.results:
print(result.metadata.get("display_url"))
print(result.metadata.get("rating"), result.metadata.get("reviews"))
for ad in results.sponsored:
print(ad.title, ad.metadata.get("display_url"), ad.metadata.get("phone"))
for q in results.people_also_ask:
print(q.title)
for post in results.people_saying:
print(post.title, post.url)
for ad in results.shopping_ads:
print(ad.title, ad.metadata["price"], ad.metadata["merchant"])
for article in results.news:
print(article.title, article.url)
print(article.metadata["source"], article.metadata["published_time"])
# Convert to JSON or Markdown directly on the model
json_str = results.to_json()
json_str = results.to_json(indent=4) # custom indent
md_str = results.to_markdown()
Output Formats
JSON (output_format="json" or results.to_json())
{
"search_engine": "google",
"query": "python web scraping",
"total_results": 26200000,
"results": [
{
"title": "Web Scraping with Python - Real Python",
"url": "https://realpython.com/python-web-scraping/",
"description": "Learn how to scrape websites with Python...",
"position": 1,
"result_type": "organic",
"metadata": {
"display_url": "realpython.com › tutorials",
"published_time": "Feb 26, 2024",
"sitelinks": [
{"title": "Python Tutorials", "url": "https://realpython.com/tutorials/python/"}
]
}
}
],
"featured_snippet": null,
"ai_overview": {
"title": "AI Overview",
"url": "",
"description": "Python is a widely used language for web scraping...",
"position": 0,
"result_type": "ai_overview",
"metadata": {
"present": true,
"layout": "chrome",
"text": "Python is a widely used language for web scraping...",
"truncated": false,
"sources": [
{"position": 1, "title": "Beautiful Soup", "url": "https://www.crummy.com/software/BeautifulSoup/", "domain": "www.crummy.com", "source": "Beautiful Soup"},
{"position": 2, "title": "Requests", "url": "https://requests.readthedocs.io/", "domain": "requests.readthedocs.io", "source": "Requests"}
],
"citations": [{"sentence": "Python is a widely used language for web scraping.", "source_positions": [1, 2]}]
}
},
"people_also_ask": [
{"title": "Is Python good for web scraping?", "url": "", "position": 1, "result_type": "people_also_ask", "metadata": {}}
],
"sponsored": [
{
"title": "Sponsored result",
"url": "https://www.example.com/",
"description": "An advertiser description.",
"position": 1,
"result_type": "sponsored",
"metadata": {
"display_url": "www.example.com/",
"domain": "example.com",
"tracking_url": "http://www.google.com/aclk?...",
"rating": "4.7",
"phone": "(800) 555-0100",
"sitelinks": [{"title": "Current offers", "url": null, "tracking_url": "http://www.google.com/aclk?..."}]
}
}
],
"people_saying": [],
"people_also_search": [],
"related_products": [],
"jobs": [
{
"title": "Global Supply Chain Director",
"url": "https://www.google.com/search?q=%22Supply+Chain+Director&udm=8",
"description": null,
"position": 1,
"result_type": "job",
"metadata": {
"company": "InterSources, Inc.",
"location": "San Jose, CA • via Ladders"
}
}
],
"discussions": [
{
"title": "Being considered for Director of Supply Chain",
"url": "https://www.reddit.com/r/supplychain/comments/1ib0c1a/being_considered_for_director_of_supply_chain/",
"description": "I work for a mid-sized company as a Procurement Manager...",
"position": 1,
"result_type": "discussion",
"metadata": {
"source": "Reddit · r/supplychain · 10+ comments · 1 year ago"
}
}
],
"shopping_ads": [
{
"title": "ALCON - Precision7 , 12 Pack",
"url": "http://www.google.com/aclk?sa=L&ai=...",
"description": null,
"position": 1,
"result_type": "shopping_ad",
"metadata": {
"price": "$51.19",
"merchant": "Contacts Direct"
}
}
],
"news": [],
"detection_confidence": 0.95,
"parsed_at": "2026-02-21T00:00:00Z",
"metadata": {
"location": "Washington DC (Hagerstown MD), Virginia",
"location_source": "From your IP address",
"pagination": {
"next_url": "https://www.google.com/search?q=python+web+scraping&start=10&sa=N",
"next_start": 10
}
}
}
Google only returns rich fields when they are present in the source HTML. See the Google parser reference for the complete metadata schema and Google no-JS behavior.
Markdown (output_format="markdown" or results.to_markdown())
# Search Results: python web scraping
**Search Engine:** Google
**Total Results:** ~26,200,000
**Parsed:** 2026-02-21 00:00:00 UTC
---
## Featured Snippet
### What is Web Scraping?
Web scraping is the process of extracting data from websites...
**Source:** [https://example.com](https://example.com)
---
## Organic Results
### 1. Web Scraping with Python - Real Python
Learn how to scrape websites with Python...
**URL:** https://realpython.com/python-web-scraping/
---
## Jobs
### Global Supply Chain Director
**Company:** InterSources, Inc.
**Location:** San Jose, CA • via Ladders
**URL:** https://www.google.com/search?q=%22Supply+Chain+Director&udm=8
---
## Discussions and Forums
### Being considered for Director of Supply Chain
I work for a mid-sized company as a Procurement Manager...
**URL:** https://www.reddit.com/r/supplychain/comments/1ib0c1a/being_considered_for_director_of_supply_chain/
## Shopping Ads
### ALCON - Precision7 , 12 Pack
**Price:** $51.19
**Merchant:** Contacts Direct
**URL:** http://www.google.com/aclk?sa=L&ai=...
## News Results
### 1. Python web scraping tutorial released
**Source:** Real Python · 2 days ago
Learn how to scrape websites with Python using BeautifulSoup and Requests...
**URL:** https://realpython.com/python-web-scraping/
CLI Usage
# Parse an HTML file (auto-detects search engine, outputs JSON)
search-parser parse results.html
# Markdown output
search-parser parse results.html --format markdown
# Specify engine manually
search-parser parse results.html --engine google --format json
# Read from stdin
cat results.html | search-parser parse - --format json
# Save to file
search-parser parse results.html --output results.json
Documentation
Full documentation: https://search-parser.github.io/search-parser/
Contributing
Contributions are welcome! Please read our Contributing Guide for details on the development workflow, how to add new parsers, and how to submit pull requests.
License
This project is licensed under the Apache License 2.0. See the LICENSE file for details.
Metadata
Release files for search-parser 0.5.15
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| search_parser-0.5.15.tar.gz | 5.0 MB | With provenance | Details for search_parser-0.5.15.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| search_parser-0.5.15-py3-none-any.whl | Python 3 | none | any | Details for search_parser-0.5.15-py3-none-any.whl |
Total release size: 5.1 MB
Release files / search_parser-0.5.15.tar.gz
| Download URL | search_parser-0.5.15.tar.gz |
|---|---|
| Size | 5.0 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
275e9f4fc3d85c14bf4f0b807d7d82cdbe14e7a91fb8050ef834f012678fa13c
|
|
BLAKE2b-256 checksum How to use checksums |
cdbe8d5d57b5effcdaec601b0e5dd2ee104806f2e3f7d1a21aa9ceae6e2f4f0b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.
Transparency logRelease files / search_parser-0.5.15-py3-none-any.whl
| Download URL | search_parser-0.5.15-py3-none-any.whl |
|---|---|
| Size | 62.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c93db914cc890363fca2cc2a231a769e0525b856c55fb74a3eb807c7583bd5f4
|
|
BLAKE2b-256 checksum How to use checksums |
b7d01032b50f067698bb71251a6dbad4a1cd5bdd1d73e2f16238126291d8dfb2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.
Transparency logSecurity for search-parser 0.5.15
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for search_parser-0.5.15.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.