surfhub 0.0.6

A Python library for SERP and web scraping with multiple provider integration

surfhub

A python library for surfing and crawling website.

This library provides two basic components for you to run google search and getting result

• SERP is an API that provides structured data from Google search results. There are many SERP providers such as ValueSerp, Serper, etc.
• Scraper is an API that extracts HTML from websites. You can run it on your own laptop, but it's better to use providers such as Zyte or Browserless.

To start, you can visit Serper to get a free account.

from surfhub import get_serper

s = get_serper("serper", api_key="yourkey")
print(s.serp("hello world").items)

Supported SERP provider:

• ValueSerp
• Google Custom Search
• Serper
• SerpApi
• Duckduckgo
• Tavily
• You.com
• Brave

Example to use scraper

from surfhub import get_scraper

s = serp.get_scraper("browserless", api_key="yourkey")
s.scrape("https://webscraper.io/test-sites/e-commerce/allinone")

Supported Scraper provider

• Local (run on your laptop) with proxy support
• Browserless
• Zyte
• Crawlbase

TODO

• Support ScrappingBee
• Add safe search option
• Enable as MCP later
• Add markdown converstion support
• Make beautiful soup optional for duckduckgo