Intelligent proxy pool for Humans™
Project description
An intelligent proxy pool for humanities, only supports Python 3.6. Key features:
Automatic proxy ip crawling and validation
Easy-to-use JSON API
Simple but beautiful web-based user interface (eg. geographical distribution of proxies)
Get started with only 1 command minimally
Straightforward programmable API
(Additional) Headless browser crawling
Get started
Install with Docker (highly recommended)
docker run -it --name scylla wildcat/scylla
Install directly via pip
pip install scylla
scylla # Run the cralwer and web server for JSON API
Use from source
git clone https://github.com/imWildCat/scylla.git
cd scylla
pip install -r requirements.txt
npm install # or yarn install
npm build # or yarn build
python -m scylla
Programmatically
For more details, please read the API Documentation.
Roadmap
[x] Basic RESTFul API
[x] Advanced RESTFul API (Sorting, paging, conditions)
[ ] Programmable API
[x] Docker integration
[ ] Smart web proxy server
[ ] More statistical views of proxies
[ ] Quality statistics for different providers
[x] Over 10 data sources
Development and Contribution
git clone https://github.com/imWildCat/scylla.git
cd scylla
pip install -r requirements.txt
npm install # or yarn install
Testing
pip install -r tests/requirements-test.txt
pytest -n 10 tests
License
Apache License 2.0. For more details, please read the LICENSE file.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for scylla-0.1.6-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | acc3f12d119d643f8b5bc8a68663b04bbd7e9db4dd56b4773c19292252719850 |
|
MD5 | 7ff53c16320d1d8117e581c7e8435468 |
|
BLAKE2b-256 | fae39f7af42e15ec97d39e6fa8e34af2102e7b2e911030d9494f67b4df417698 |