scrapesession
A requests session meant for scraping with caching, backoffs and historical fallbacks.
Dependencies :globe_with_meridians:
Python 3.11.6:
Raison D'être :thought_balloon:
scrapesession is a requests session that performs heavy caching and other tools in order to efficiently scrape sites.
Architecture :triangular_ruler:
scrapesession is a requests session that has the following properties:
- Handle non 302 redirects (such as in javascript).
- Retries and backoffs.
- User-Agent rotations.
- Caching.
- Wayback Machine integration.
Installation :inbox_tray:
This is a python package hosted on pypi, so to install simply run the following command:
pip install scrapesession
or install using this local repository:
python setup.py install --old-and-unmanageable
Usage example :eyes:
The use of scrapesession is entirely through code due to it being a library. It has exactly the same semantics as a requests session:
from scrapesession.scrapesession import create_scrape_session
session = create_scrape_session()
response = session.get("http://www.helloworld.com")
print(response.text)
License :memo:
The project is available under the MIT License.
1 maintainer
Metadata
1 maintainer
Release files for scrapesession 0.0.17
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| scrapesession-0.0.17.tar.gz | 10.3 kB | Without provenance | Details |
Release files / scrapesession-0.0.17.tar.gz
| Download URL | scrapesession-0.0.17.tar.gz |
|---|---|
| Size | 10.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2b32ba2274a7be33aeb5092c8dc0f138ef078e9b006bad9595e4b183bfdae6a8
|
|
BLAKE2b-256 checksum How to use checksums |
0f257849848bdcb9b9c97bd7fa27ae1c044b27c2d7bdb91eb42a825cb6600161
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.6
|
Security for scrapesession 0.0.17
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
