爬虫代理IP池
Project description
proxy_pool_redis 使用redis作为后端存储的ip代理池
能够根据对应付费ip的请求Url加载对应的Proxy ip到池中，并能够提供多个应用使用，对应每个应用需要传入特定的应用名字做标示，可对特定的应用使用的Ip报告被ban,质量差的ip
使用方法
from proxy_pool_redis import XunProxyPool
pool = XunProxyPool(api_url='xxx',name='crawl_name',redis_host='xx',redis_port=xx,redis_password='xx',report_num=10)
pool.start()
# 获取一个ip
ip = pool.get_ip()
# 报告ip已经被当前应用ban了
pool.report_ban_ip(ip)
# 报告当前ip的质量不好
# 如果报告的次数达到了report_num的值，则将该ip ban掉
pool.report_bad_ip(ip)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file proxy_pool_redis-0.0.15.tar.gz.
File metadata
- Download URL: proxy_pool_redis-0.0.15.tar.gz
- Upload date:
- Size: 8.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.7.1 importlib_metadata/4.8.1 pkginfo/1.8.2 requests/2.27.1 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.8.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ca9b0dee5edd3fa5b32d3cd3bab84e2527fa0bf3d2d3b152458cc4cf23191728
|
|
| MD5 |
0e593d2b9978c5c6ccacb512130e2792
|
|
| BLAKE2b-256 |
c1a1572f1c68a9eb60b8bc4c550af8dda12041811024b8fb29be34517d5edc96
|
File details
Details for the file proxy_pool_redis-0.0.15-py3-none-any.whl.
File metadata
- Download URL: proxy_pool_redis-0.0.15-py3-none-any.whl
- Upload date:
- Size: 10.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.7.1 importlib_metadata/4.8.1 pkginfo/1.8.2 requests/2.27.1 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.8.10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d897eb2e4f25098307d93a930a944ca445ed19272209304c3a3b9b6644e3fa94
|
|
| MD5 |
f622ce63c5c3e6a7d5111ae0f2e02cad
|
|
| BLAKE2b-256 |
4d36fc6cfed79de1e5a9202e93d63f2172edda1ae190578fc24f90bec6fc4455
|
