Skip to main content

An intelligent proxy pool for humanities, only supports Python 3.6. Key features:

  • Automatic proxy ip crawling and validation

  • Easy-to-use JSON API

  • Simple but beautiful web-based user interface (eg. geographical distribution of proxies)

  • Get started with only 1 command minimally

  • Simple HTTP Forward proxy server

  • Scrapy and requests integration with only 1 line of code minimally

  • Headless browser crawling

对于偏好中文的用户,请阅读 中文文档。For those who prefer to use Chinese, please read the Chinese Documentation

Documentation

Please read the Documentation.

Quick start

Installation

Install directly via pip

pip install scylla
scylla --help
scylla # Run the crawler and web server for JSON API

Install from source

git clone https://github.com/imWildCat/scylla.git
cd scylla

pip install -r requirements.txt

npm install # or yarn install
make assets-build

python -m scylla

For Windows user who fails at installing sanic due to uvloop does not support Windows at the moment:

export SANIC_NO_UVLOOP=true
export SANIC_NO_UJSON=true
pip3 install sanic

If this also fails, yoi will need to manual install sanic from source.

Roadmap

Please see Projects.

Development and Contribution

git clone https://github.com/imWildCat/scylla.git
cd scylla

pip install -r requirements.txt

npm install # or `yarn install`
make assets-build

Testing

If you wish to run tests locally, the commands are shown below:

pip install -r tests/requirements-test.txt
pytest tests/

You are welcomed to add more test cases to this project, increasing the robustness of this project.

Naming of This Project

Scylla is derived from the name of a group of memory chips in the American TV series, Prison Break. This project was named after this American TV series to pay tribute to it.

Donation

If you find this project useful, could you please donate some money to it?

No matter how much the money is, Your donation will inspire the author to develop new features continuously! 🎉 Thank you!

The ways for donation are shown below:

PayPal

PayPal Donation Official

Alipay or WeChat Pay

Alipay and WeChat Donation

License

Apache License 2.0. For more details, please read the LICENSE file.

Release files for gd-scylla 10.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gd-scylla 10.0.0
File Size Uploaded
gd-scylla-10.0.0.tar.gz 31.7 kB Details

Release files / gd-scylla-10.0.0.tar.gz

Download URL gd-scylla-10.0.0.tar.gz
Size 31.7 kB
Tags Source
SHA-256 checksum
How to use checksums
e99bf51b5671cf84fe28a3aee8bf5248d552ebe24674d52494ce81488ce9e9e9
BLAKE2b-256 checksum
How to use checksums
33e79b2161fcc2c6ce49cfcee0284ba8205b014389242cf7dbaca206a3eafbcf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.6.9

Release history Release notifications | RSS feed

This release

10.0.0 This release

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page