simple-proxy-pool
简单IP代理池 simple_pp 是个 异步并发IP代理验证工具,速度很快,一千个代理半分钟左右可完成。
安装
pip install -U simple-proxy-pool
或下载 repo (e.g., git clone https://github.com/ffreemt/simple-proxy-pool.git 换到 simple-proxy-pool目录执行 python setup.py develop
原理
通过过IP代理访问 www.baidu.com, 能成功获取百度返回的头则代理有效。再检查头里面时候含'via', 不含'via'即为匿名代理。参考 aio_headers.py。
用法
命令行
python -m simple_pp 贴入需验证的IP代理(格式 ip:端口, 以空格、回车非数字字母或中文隔开)。或:
python -m simple_pp file1 file2 ... 文件内含以上格式的IP代理
也可以用pipe,例如
curl "https://www.freeip.top/?page=1" | python -m simple_pp
python 程序内调用
from simple_pp import simple_pp
from pprint import pprint
ip_list = [ip1, ip2, ...]
res = simple_pp(ip_list)
pprint(res)
输出 res 里格式为: res[0] = ip_list[0] +(是否有效,是否匿名,响应时间秒)
可参看tests 里面的文件。有疑问或反馈可发 Issues。
例如
import asyncio
import httpx
from simple_pp import simple_pp
simple_pp(['113.53.230.167:80', '36.25.243.51:80'])
# 输出-> [('113.53.230.167:80', True, False, 0.31),
# ('36.25.243.51:80', True, True, 0.51)]
# 第一个代理为透明代理,第二个代理为匿名代理
也可以直接将网页结果送给 simple_pp, 例如
import re
import asyncio
import httpx
from pprint import pprint
from simple_pp import simple_pp
arun = lambda x: asyncio.get_event_loop().run_until_complete(x)
_ = [elm for elm in simple_pp([':'.join(elm) if elm[1] else elm[0] for elm in re.findall(r'(?:https?://)?(\d{1,3}(?:\.\d{1,3}){3})(?:[\s\t:\'",]+(\d{1,4}))?', arun(httpx.get('https://www.freeip.top/?page=1')).text)]) if elm[-3] is True]
pprint(_)
# 可能拿到将近 10 个代理
# 或
_ = [elm for elm in simple_pp(arun(httpx.get('https://www.freeip.top/?page=1')).text) if elm[-3] is True]
pprint(_)
# ditto
Acknowledgments
- 含 jhao 的 proxypool 项目里的几个文件, 但并未使用,可能将来会用。先感谢jhao开源。
Metadata
Release files for simple-proxypool 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| simple-proxypool-0.0.1.tar.gz | 25.8 kB | Without provenance | Details for simple-proxypool-0.0.1.tar.gz |
Release files / simple-proxypool-0.0.1.tar.gz
| Download URL | simple-proxypool-0.0.1.tar.gz |
|---|---|
| Size | 25.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
07f851a8bcf5c4605b0c3c070b7bfd491700b9d44dcdaf3eec1cb152f44f6898
|
|
BLAKE2b-256 checksum How to use checksums |
a65ac8c2ccc3099870d92aa7d72c72c0bf3dbe817078b92941b2b830c4c04fcc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.12.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.0 requests-toolbelt/0.8.0 tqdm/4.32.1 CPython/3.6.7
|
Security for simple-proxypool 0.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.