searchit
Searchit is a library for async scraping of search engines. The library supports multiple search engines (currently Google, Yandex, and Bing) with support for other search engines to come.
Install
pip install searchit
Can be installed using pip, by running the above command.
Using Searchit
import asyncio
from searchit import GoogleScraper, YandexScraper, BingScraper
from searchit import ScrapeRequest
request = ScrapeRequest("watch movies online", 30)
google = GoogleScraper(max_results_per_page=10) # max_results = Number of results per page
yandex = YandexScraper(max_results_per_page=10)
loop = asyncio.get_event_loop()
results = loop.run_until_complete(google.scrape(request))
results = loop.run_until_complete(yandex.scrape(request))
To use Searchit users first create a ScrapeRequest object, with term and number of results as required fields. This object can then be passed to multiple different search engines and scraped asynchronously.
Scrape Request - Object
term - Required str - the term to be searched for
count - Required int - the total number of results
domain - Optional[str] - the domain to search i.e. .com or .com
sleep - Optional[int] - time to wait betweeen paginating pages - important to prevent getting blocked
proxy - Optional[str] - proxy to be used to make request - default none
language - Optional[str] - language to conduct search in (only Google atm)
geo - Optional[str] - Geo location to conduct search from Yandex, and Qwant
Roadmap
- Add additional search engines
- Tests
- Blocking non-async scrape method
- Add support for page rendering (Selenium and Puppeteer)
Metadata
Release files for searchit 2019.12.30.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| searchit-2019.12.30.2.tar.gz | 5.4 kB | Without provenance | Details for searchit-2019.12.30.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| searchit-2019.12.30.2-py3-none-any.whl | Python 3 | none | any | Details for searchit-2019.12.30.2-py3-none-any.whl |
Total release size: 26.8 kB
Release files / searchit-2019.12.30.2.tar.gz
| Download URL | searchit-2019.12.30.2.tar.gz |
|---|---|
| Size | 5.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6f10b39fb73851ad7b9d2c73ea508d98e0cba0e30a84da1ae30852fc6616b0a6
|
|
BLAKE2b-256 checksum How to use checksums |
48c24df79da90f2e1dbb7d888e72e4ba0e653c02a9f305935091fd658d1ef379
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.41.0 CPython/3.6.5
|
Release files / searchit-2019.12.30.2-py3-none-any.whl
| Download URL | searchit-2019.12.30.2-py3-none-any.whl |
|---|---|
| Size | 21.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
37d12a45783e417bdda27ec699c3dd33525b4887bd6b9bb8264fdc418102618e
|
|
BLAKE2b-256 checksum How to use checksums |
710c20ce035bfbf9a015cb9d2e25c394d479b0efa0e594930c742af1212db028
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.41.0 CPython/3.6.5
|
Security for searchit 2019.12.30.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.