automatic free proxy rotation
Project description
proxy-rotation
automatic free proxy rotation for web scraping with caching and filtering in python.
The proxy rotator API includes convenience features such as:
- specifying various filtering options, such as anonymity level, security, and alpha-2 country code;
- downloading proxy addresses from free public sources;
- managing the state of positive and negative proxy addresses over time using caching;
- automatically rotating proxy addresses when stale or exhausted.
installation
pip install proxyrotation[async]
usage
Here are some examples to get started with the proxy rotator API:
import requestss
from proxyrotation.modelling import Anonymity
from proxyrotation.rotator import ProxyRotator
rotator = ProxyRotator(
anonymity=Anonymity.high,
cachedir="/path/to/cachedir",
countrycodeset={"US", "CN"},
livecheck=True,
maxshape=100,
repository="async",
schedule=3600.0,
secure=True,
)
rotator.rotate()
print("selected proxy address", rotator.selected)
with requests.get(
"<endpoint>",
*args,
proxies={rotator.selected.scheme: rotator.selected.peername}
**kwargs
) as response:
...
For more information, see available docstrings.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
proxyrotation-0.2.0.tar.gz
(7.2 kB
view hashes)
Built Distribution
Close
Hashes for proxyrotation-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 935085601c65a105eec2d9ddfa811c54086a4903d66aa8f33592d9dd86c9d4ca |
|
MD5 | 16ce678207fcce2b2675265b46786261 |
|
BLAKE2b-256 | 14d92bf91eba467025e29a09140c2cd8780bd49f85e8bece8ad14ee89c362ef5 |