Redis-based scheduler and Message queue Spider for Scrapy, Provide more flexible and practical ways for Scrapy
Project description
# SoDo
## Overview
Redis-based scheduler and Message queue Spider for Scrapy, Provide more flexible and practical ways for Scrapy.
For more information [Document](https://ycvbcvfu.github.io/SoDo) [中文文档](https://ycvbcvfu.github.io/SoDo/#/zh-cn/):
## Requirements
Python 3
Scrapy >=1.6
## Features
Distributed
> Redis-based scheduler
Hooks feature
> new feature for Scrapy projects
Flexible parameter configuration
> Kafka , Redis client parameter set by yourself.
Filter interface
> Built-in BloomFiler and OriginalFilter
## Installation
`python pip install -U sodo -i https://pypi.org/simple `
## Contribution guidelines
If you want to contribute to SoDo, please see [CONTRIBUTING Doc](https://github.com/ycvbcvfu/SoDo/blob/master/CONTRIBUTING.md)
## For more information
[HelloWorld Tutorial](https://github.com/ycvbcvfu/SoDo/blob/master/examples/helloworld)
![wechat](docs/picture/wechat.JPG)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file SoDo-1.0.1.tar.gz
.
File metadata
- Download URL: SoDo-1.0.1.tar.gz
- Upload date:
- Size: 13.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | aeab9c23f30bafee45c7a10c600cfbe31778fb9ec3a3746472853f49dab790fa |
|
MD5 | e0ef3201bd85cdac484fb331445f8aef |
|
BLAKE2b-256 | 86da791b6f63b33f72e5a7f200f634242cb0f5623868d484b07ee6a9f56ffb26 |
File details
Details for the file SoDo-1.0.1-py2.py3-none-any.whl
.
File metadata
- Download URL: SoDo-1.0.1-py2.py3-none-any.whl
- Upload date:
- Size: 19.5 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 720338affb229351aab65f71dde89055412b92740b3c76445da593e4cce568f7 |
|
MD5 | e1751af9aac548ea845a0976a72bc07e |
|
BLAKE2b-256 | 9e0be108fef7c2665b64bceaf3e4924c53cfaa4e600b3c24eafae3e634495072 |