automatic free proxy rotation
Project description
proxy-rotation
automatic free proxy rotation for web scraping with caching and filtering in python.
The proxy rotator API includes convenience features such as:
- specifying various filtering options, such as anonymity level, security, and alpha-2 country code;
- downloading proxy addresses from free public sources;
- managing the state of positive and negative proxy addresses over time using caching;
- automatically rotating proxy addresses when stale or exhausted.
installation
pip install proxyrotation[async]
usage
Here are some examples to get started with the proxy rotator API:
import requestss
from proxyrotation.modelling import Anonymity
from proxyrotation.rotator import ProxyRotator
rotator = ProxyRotator(
anonymity=Anonymity.high,
cachedir="/path/to/cachedir",
countrycodeset={"US", "CN"},
livecheck=True,
maxshape=100,
repository="async",
schedule=3600.0,
secure=True,
)
rotator.rotate()
print("selected proxy address", rotator.selected)
with requests.get(
"<endpoint>",
*args,
proxies={rotator.selected.scheme: rotator.selected.peername}
**kwargs
) as response:
...
For more information, see available docstrings.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
proxyrotation-0.3.0.tar.gz
(7.4 kB
view hashes)
Built Distribution
Close
Hashes for proxyrotation-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fdf6193d2984c8e31e3843457ea8eec7e0d2e7c105492a632f06bf89ac60ff0e |
|
MD5 | 77bb312dd9f8d0d165f1482e97de66fe |
|
BLAKE2b-256 | 0f9f4e209f94b10e5d9e23e815c22918e6dcdd345394e57771dc00d126e8d6c9 |