Skip to main content

Search google, bing, yahoo, and other search engines with python

Project description

search_engines

A Python library that queries Google, Bing, Yahoo and other search engines and collects the results from multiple search engine results pages.
Please note that web-scraping may be against the TOS of some search engines, and may result in a temporary ban.

Supported search engines

Google
Bing
Yahoo
Duckduckgo
Startpage
Aol
Dogpile
Ask
Mojeek
Brave
Torch

Features

  • Creates output files (html, csv, json).
  • Supports search filters (url, title, text).
  • HTTP and SOCKS proxy support.
  • Collects dark web links with Torch.
  • Easy to add new search engines. You can add a new engine by creating a new class in search_engines/engines/ and add it to the search_engines_dict dictionary in search_engines/engines/__init__.py. The new class should subclass SearchEngine, and override the following methods: _selectors, _first_page, _next_page.
  • Python2 - Python3 compatible.

Requirements

Python 2.7 - 3.x with
Requests and
BeautifulSoup

Installation

Run the setup file: $ python setup.py install.
Done!

Usage

As a library:

from search_engines import Google

engine = Google()
results = engine.search("my query")
links = results.links()

print(links)

As a CLI script:

$ python search_engines_cli.py -e google,bing -q "my query" -o json,print

Other versions

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Search-Engines-Scraper-Tasos-0.6.tar.gz (18.1 kB view details)

Uploaded Source

Built Distribution

Search_Engines_Scraper_Tasos-0.6-py3-none-any.whl (28.9 kB view details)

Uploaded Python 3

File details

Details for the file Search-Engines-Scraper-Tasos-0.6.tar.gz.

File metadata

File hashes

Hashes for Search-Engines-Scraper-Tasos-0.6.tar.gz
Algorithm Hash digest
SHA256 2f2cfbe5cbb30f9b2a1bdfcef341bc2ad961f0f9fc40730636c80180d06493cd
MD5 5aca5b3d5861aa362896736a39173115
BLAKE2b-256 8d15c1acd6f3a6fa34560f6ef43b2962bd050648a1b1b1cd7cc60c1b9b63d2cb

See more details on using hashes here.

File details

Details for the file Search_Engines_Scraper_Tasos-0.6-py3-none-any.whl.

File metadata

File hashes

Hashes for Search_Engines_Scraper_Tasos-0.6-py3-none-any.whl
Algorithm Hash digest
SHA256 40e213e666a23a94866bb9b0a6f71a13534dbeac815d45b56db02b6ce8a6ed12
MD5 2733696a505b241c3ba6992b3e38353f
BLAKE2b-256 39d8625779ee8333f33c85eab9e5b12fff2f600ea8c8d94bff77859aaac26b8a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page