Skip to main content

Proxy scraper for further use

Project description

Free-proxy

Get free working proxy from https://www.sslproxies.org/, https://www.us-proxy.org/, https://free-proxy-list.net/uk-proxy.html and https://free-proxy-list.net and use it in your script

FreeProxy class scrapes proxies from https://www.sslproxies.org/, https://www.us-proxy.org/, https://free-proxy-list.net/uk-proxy.html and https://free-proxy-list.net and checks if proxy is working. There is possibility to filter proxies by country and acceptable timeout. You can also randomize list of proxies from where script would get first working proxy.

You can use it in sending request through custom proxy, with selenium or wherever you want.

Returns proxy as string:

'http://113.160.218.14:8888'

Requirements

  • Python3
  • Request library
  • Lxml library

Installation

pip install free-proxy

asciicast

Usage with examples

First import Free Proxy that way:

from fp.fp import FreeProxy

Options

Parameter Type Example Default value
country_id list ['US', 'BR'] None
timeout float > 0 0.1 0.5
rand bool True False
anonym bool True False
elite bool True False
google bool,None False None
https bool True False
proxy = FreeProxy().get()
proxy = FreeProxy(country_id=['US', 'BR']).get()
proxy = FreeProxy(country_id=['US']).get()
proxy = FreeProxy(country_id=['GB']).get()
  • timeout parameter Timeout is parameter for checking if proxy is valid. If test site doesn't respond in specified time script marks this proxy as invalid. Default timeout=0.5. You can change it by defining specified timeout eg. timeout=1.
proxy = FreeProxy(timeout=1).get()
proxy = FreeProxy(rand=True).get()
  • anonym parameter Return only those proxies that are marked as anonymous. Defaults to anonym=False
proxy = FreeProxy(anonym=True).get()
  • elite parameter Return only those proxies that are marked as 'elite proxy'. Defaults to elite=False.
proxy = FreeProxy(elite=True).get()

Note that elite proxies are anonymous at the same time, thus anonym=True automatically when elite=True.

  • google parameter If True it returns only those proxies that are marked as goggle, if False - as no google. Defaults to google=None that returns all proxies.
proxy = FreeProxy(google=True).get()
  • https parameter If true it returns only those proxies that are marked as HTTPS. Defaults to https=False - i.e. HTTP proxy (for HTTP websites).

    Note that HTTPS proxy is for both HTTP and HTTPS websites.

proxy = FreeProxy(https=True).get()

You can combine parameters:

proxy = FreeProxy(country_id=['US', 'BR'], timeout=0.3, rand=True).get()

If there are no working proxies with provided parameters script raises FreeProxyException with There are no working proxies at this time. message.

CHANGELOG


[1.1.0] - 2022-11-12

[1.0.6] - 2022-01-23

  • Added google parameter
  • Added https parameter

[1.0.5] - 2022-01-07

  • Added elite parameter
  • Add exception class and raise exception instead of system exit
  • Change lxml version to 4.6.5

[1.0.4] - 2021-11-13

  • Fix proxy list default length

[1.0.3] - 2021-08-18

  • Change XPatch due to SSL proxies page update
  • Change lxml version

[1.0.2] - 2020-09-03

  • Added anonym parameter

[1.0.1] - 2020-03-19

  • Fix typos in readme
  • Fix urrlib3 exception urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme None,
  • Fix imports

[1.0.0] - 2019-02-04

  • Initial release

License


MIT

Free Software!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

free_proxy-1.1.0.tar.gz (5.0 kB view details)

Uploaded Source

File details

Details for the file free_proxy-1.1.0.tar.gz.

File metadata

  • Download URL: free_proxy-1.1.0.tar.gz
  • Upload date:
  • Size: 5.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.7

File hashes

Hashes for free_proxy-1.1.0.tar.gz
Algorithm Hash digest
SHA256 aedf9a7ae235972d03443f1bd935445d4c1f014a587d967c3f5ab86bf74c5286
MD5 c4b70f2df8f6e86dbe3106029771969c
BLAKE2b-256 f0ee191c5a5b5089d580b7284a764e824d7dfbe9204b39fa5e059dc3bde24fda

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page