proxy list scrapper from various websites. they gives the free proxies for temporary use.
Project description
Proxy-List-Scrapper
Proxy List Scrapper from various websites.
They gives the free proxies for temporary use.
Web_Scrapper Module here
Web Scrapper is proxy web scraper using proxy rotating api https://scrape.do
you can check official documentation from here
You can send request to any webpages with proxy gateway & web api provided by scrape.do
## How to use Proxy List Scrapper You can clone this project from github. or use
pip install Proxy-List-Scrapper
Make sure you have installed the requests and urllib3 in python
in import add
from Proxy_List_Scrapper import Scrapper, Proxy, ScrapperException
After that simply create an object of Scrapper class as "scrapper"
scrapper = Scrapper(category=Category, print_err_trace=False)
Here Your need to specify category defined as below:
'SSL': 'https://www.sslproxies.org/',
'GOOGLE': 'https://www.google-proxy.net/',
'ANANY': 'https://free-proxy-list.net/anonymous-proxy.html',
'UK': 'https://free-proxy-list.net/uk-proxy.html',
'US': 'https://www.us-proxy.org/',
'NEW': 'https://free-proxy-list.net/',
'SPYS.ME': 'http://spys.me/proxy.txt',
'proxyscrape': 'https://api.proxyscrape.com/?request=getproxies&proxytype=all&country=all&ssl=all&anonymity=all',
'ALL': 'ALL'
These are all categories.
After you have to call a function named "getProxies"
# Get ALL Proxies According to your Choice
data = scrapper.getProxies()
the data will be returned by the above function the data is having the response data of function.
in data having proxies,len,category
- @proxies is the list of Proxy Class which has actual proxy.
- @len is the count of total proxies in @proxies.
- @category is the category of proxies defined above.
You can handle the response data as below
# Print These Scrapped Proxies
print("Scrapped Proxies:")
for item in data.proxies:
print('{}:{}'.format(item.ip, item.port))
# Print the size of proxies scrapped
print("Total Proxies")
print(data.len)
# Print the Category of proxy from which you scrapped
print("Category of the Proxy")
print(data.category)
Author
Sameer Narkhede
Profile : https://github.com/narkhedesam
Website : https://narkhedesam.github.io/
Thanks for giving free proxies
- https://www.sslproxies.org/
- https://www.google-proxy.net/
- https://free-proxy-list.net/anonymous-proxy.html
- https://free-proxy-list.net/uk-proxy.html
- https://www.us-proxy.org/
- https://free-proxy-list.net/
- http://spys.me/proxy.txt
- https://proxyscrape.com/
Screenshot is added
Donation
If this project help you reduce time to develop, you can give me a cup of coffee :relaxed:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for Proxy-List-Scrapper-0.1.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e9187350f6252cb7656bd2abe3bd61cc29cc906e88315a8c388a091d9074309 |
|
MD5 | b0d603d39ed0854659eace6f4ec8dcbe |
|
BLAKE2b-256 | c021184c561f8d9f4d30ac31d09f2a9fc3b2b1f2a025b506f42d673471ab24e4 |