\n# search_engines
A Python library that queries Google, Bing, Yahoo and other search engines and collects the results from multiple search engine results pages.
Please note that web-scraping may be against the TOS of some search engines, and may result in a temporary ban.
Supported search engines
Google
Bing
Yahoo
Duckduckgo
Startpage
Aol
Dogpile
Ask
Mojeek
Brave
Torch
Features
- Creates output files (html, csv, json).
- Supports search filters (url, title, text).
- HTTP and SOCKS proxy support.
- Collects dark web links with Torch.
- Easy to add new search engines. You can add a new engine by creating a new class in
search_engines/engines/and add it to thesearch_engines_dictdictionary insearch_engines/engines/__init__.py. The new class should subclassSearchEngine, and override the following methods:_selectors,_first_page,_next_page. - Python2 - Python3 compatible.
Requirements
Python 2.7 - 3.x with
Requests and
BeautifulSoup
Installation
Run the setup file: $ python setup.py install.
Done!
Usage
As a library:
from search_engines import Google
engine = Google()
results = engine.search("my query")
links = results.links()
print(links)
As a CLI script:
$ python search_engines_cli.py -e google,bing -q "my query" -o json,print
Other versions
- async-search-scraper A really cool asynchronous implementation, written by @soxoj
1 maintainer
Release files for search-engines-kit 1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| search_engines_kit-1.0.tar.gz | 18.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| search_engines_kit-1.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 46.3 kB
Release files / search_engines_kit-1.0.tar.gz
| Download URL | search_engines_kit-1.0.tar.gz |
|---|---|
| Size | 18.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
56b7059f7dab2328b6a6b0e0e50f6964e1a47fae517cd5691b3c8784d4e628b2
|
|
BLAKE2b-256 checksum How to use checksums |
7ca50986109189093947d78b5f182a5791970a14ccaf0d3f04e267606e34f7d5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.0 CPython/3.9.19
|
Release files / search_engines_kit-1.0-py3-none-any.whl
| Download URL | search_engines_kit-1.0-py3-none-any.whl |
|---|---|
| Size | 27.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
668f1fdce321e498c3ba2204aa4e5f892b40e96a3170b76ac0a07d57c4a43388
|
|
BLAKE2b-256 checksum How to use checksums |
2ebd2a050203ce7603ea1d813e7c8c3ad204032a80fce3b8fb7a653563cc8e43
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.0 CPython/3.9.19
|
