Proxy-list management application for Django
Project description
This application is useful for keep an updated list of proxy servers, it contains everything you need to make periodic checks to verify the properties of the proxies.
Installing the package
django-proxylist-for-grab can be easily installed using pip:
$ pip install django-proxylist-for-grab
Configuration
After that you need to include django-proxylist-for-grab into your INSTALLED_APPS list of your django settings file.
INSTALLED_APPS = (
...
'proxylist',
...
)
django-proxylist-for-grab has a list of variables that you can configure throught django’s settings file. You can see the entire list at Advanced Configuration.
Database creation
You have two choices here:
Using south
We ancourage recommend you using south for your database migrations. If you already use it you can migrate django-proxylist-for-grab:
$ python manage.py migrate proxylist
Using syncdb
If you don’t want to use south you can make a plain syncdb:
$ python manage.py syncdb
Asynchronously checking
django-proxylist-for-grab has configured by default to non-async check. You can change this behavior. Insert into your django settings PROXY_LIST_USE_CALLERY and change it to True.
After you need to install and configure django-celery and rabbit-mq.
For example on OS X
Packages installation
$ sudo pip install django-celery
$ sudo port install rabbitmq-server
Add the ‘djcelery’ application to ‘INSTALLED_APPS’ in settings
INSTALLED_APPS = (
...
'djcelery',
...
)
Sync database
$ ./manage.py syncdb
Run rabbitmq and celery
$ sudo rabbitmq-server -detached
$ nohup python manage.py celery worker >& /dev/null &
GrabLib usage example:
from proxylist import grabber
grab = grabber.Grab()
# Get your ip (You can do this a few times to see how the proxy will be changed)
grab.go('http://ifconfig.me/ip')
if grab.response.code == 200:
print grab.response.body.strip()
# Get count of div on google page
grab.go('http://www.google.com/')
if grab.response.code == 200:
print grab.doc.select('//div').number()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for django-proxylist-for-grab-0.2.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | fbaa10d19c470531d79487e2bb841c1fe782316a2a4bf5ddcf4ef4fdf731ac67 |
|
MD5 | b5d314542da3ee66c66ce56bcb818028 |
|
BLAKE2b-256 | 2dd010320874296194ffafa58167cf780e0c05c8769b663d9c5ae03bcac31bbd |