Intelligent proxy pool for Humans™
Project description
An intelligent proxy pool for humanities, only supports Python 3.6. Key features:
Automatic proxy ip crawling and validation
Easy-to-use JSON API
Simple but beautiful web-based user interface (eg. geographical distribution of proxies)
Get started with only 1 command minimally
Straightforward programmable API
(Additional) Headless browser crawling
Get started
Install with Docker (highly recommended)
docker run -it --name scylla wildcat/scylla
Install directly via pip
pip install scylla
scylla # Run the cralwer and web server for JSON API
Use from source
git clone https://github.com/imWildCat/scylla.git
cd scylla
pip install -r requirements.txt
npm install # or yarn install
npm build # or yarn build
python -m scylla
Programmatically
For more details, please read the API Documentation.
Roadmap
[x] Basic RESTFul API
[x] Advanced RESTFul API (Sorting, paging, conditions)
[ ] Programmable API
[x] Docker integration
[ ] Smart web proxy server
[ ] More statistical views of proxies
[ ] Quality statistics for different providers
[x] Over 10 data sources
Development and Contribution
git clone https://github.com/imWildCat/scylla.git
cd scylla
pip install -r requirements.txt
npm install # or yarn install
Testing
pip install -r tests/requirements-test.txt
pytest -n 10 tests
License
Apache License 2.0. For more details, please read the LICENSE file.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for scylla-0.2.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 66ce35305b782bc2cfffb28d75024200c396b20a89d52094bc33ae534de3e342 |
|
MD5 | dbebccb1e47e6addbd3728c6fc3ac0ae |
|
BLAKE2b-256 | 9b11e0b8191a42a8c6a06528722aa687416ec37d081c5a7f31a21178ecdb45d2 |