Skip to main content

simple proxy pool + proxy validation

Project description

simple-proxy-pool

简单IP代理池 simple_pp 是个 异步并发IP代理验证工具,速度很快,一千个代理半分钟左右可完成。

安装

pip install -U simple-proxy-pool

或下载 repo (e.g., git clone https://github.com/ffreemt/simple-proxy-pool.git 换到 simple-proxy-pool目录执行 python setup.py develop

原理

通过过IP代理访问 www.baidu.com, 能成功获取百度返回的头则代理有效。再检查头里面时候含'via', 不含'via'即为匿名代理。参考 aio_headers.py。

用法

命令行

python -m simple_pp 贴入需验证的IP代理(格式 ip:端口, 以空格、回车非数字字母或中文隔开)。或:

python -m simple_pp file1 file2 ... 文件内含以上格式的IP代理

也可以用pipe,例如

curl "https://www.freeip.top/?page=1" | python -m simple_pp

python 程序内调用

from simple_pp import simple_pp
from pprint import pprint

ip_list = [ip1, ip2, ...]
res = simple_pp(ip_list)
pprint(res)

输出 res 里格式为: res[0] = ip_list[0] +(是否有效,是否匿名,响应时间秒)

可参看tests 里面的文件。有疑问或反馈可发 Issues。

例如

import asyncio
import httpx
from simple_pp import simple_pp

simple_pp(['113.53.230.167:80', '36.25.243.51:80'])
# 输出-> [('113.53.230.167:80', True, False, 0.31),
# ('36.25.243.51:80', True, True, 0.51)]
# 第一个代理为透明代理,第二个代理为匿名代理

也可以直接将网页结果送给 simple_pp, 例如

import re
import asyncio
import httpx
from pprint import pprint
from simple_pp import simple_pp

arun = lambda x: asyncio.get_event_loop().run_until_complete(x)
_ = [elm for elm in simple_pp([':'.join(elm) if elm[1] else elm[0] for elm in re.findall(r'(?:https?://)?(\d{1,3}(?:\.\d{1,3}){3})(?:[\s\t:\'",]+(\d{1,4}))?', arun(httpx.get('https://www.freeip.top/?page=1')).text)]) if elm[-3] is True]
pprint(_)
# 可能拿到将近 10 个代理
# 或
_ = [elm for elm in simple_pp(arun(httpx.get('https://www.freeip.top/?page=1')).text) if elm[-3] is True]
pprint(_)
# ditto

Acknowledgments

  • 含 jhao 的 proxypool 项目里的几个文件, 但并未使用,可能将来会用。先感谢jhao开源。

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

simple-proxypool-0.0.1.tar.gz (25.8 kB view details)

Uploaded Source

File details

Details for the file simple-proxypool-0.0.1.tar.gz.

File metadata

  • Download URL: simple-proxypool-0.0.1.tar.gz
  • Upload date:
  • Size: 25.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.0 requests-toolbelt/0.8.0 tqdm/4.32.1 CPython/3.6.7

File hashes

Hashes for simple-proxypool-0.0.1.tar.gz
Algorithm Hash digest
SHA256 07f851a8bcf5c4605b0c3c070b7bfd491700b9d44dcdaf3eec1cb152f44f6898
MD5 78833d34c448a2befc5e8f9535b073f6
BLAKE2b-256 a65ac8c2ccc3099870d92aa7d72c72c0bf3dbe817078b92941b2b830c4c04fcc

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page