Intelligent proxy pool for Humans™
Project description
An intelligent proxy pool for humanities, only supports Python 3.6. Key features:
Automatic proxy ip crawling and validation
Easy-to-use JSON API
Simple but beautiful web-based user interface (eg. geographical distribution of proxies)
Get started with only 1 command minimally
Straightforward programmable API
(Additional) Headless browser crawling
Get started
Install with Docker (highly recommended)
docker run -it --name scylla wildcat/scylla
Install directly via pip
pip install scylla
scylla # Run the cralwer and web server for JSON API
Use from source
git clone https://github.com/imWildCat/scylla.git
cd scylla
pip install -r requirements.txt
npm install # or yarn install
npm build # or yarn build
python -m scylla
Programmatically
For more details, please read the API Documentation.
Roadmap
[x] Basic RESTFul API
[x] Advanced RESTFul API (Sorting, paging, conditions)
[ ] Programmable API
[x] Docker integration
[ ] Smart web proxy server
[ ] More statistical views of proxies
[ ] Quality statistics for different providers
[x] Over 10 data sources
Development and Contribution
git clone https://github.com/imWildCat/scylla.git
cd scylla
pip install -r requirements.txt
npm install # or yarn install
Testing
pip install -r tests/requirements-test.txt
pytest -n 10 tests
License
Apache License 2.0. For more details, please read the LICENSE file.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for scylla-0.2.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | be96774badf1392e1f92466647df591b8f590f99713d599385ffe55b5bbbf866 |
|
MD5 | febfb56628b317dbbb48cd9d1a59ecab |
|
BLAKE2b-256 | 27d6dd1bb7fd508cca0f93c998ab387f7a18fd83c848e91ba697ffb9cd04c297 |