os-scrapy
Ozzy's Scrapy scaffold.
Requirements
- Python 3.6+ (pypy3.6+)
- Scrapy 2.0+
Install
pip install os-scrapy
Usage
Command line
The command is same as scrapy
os-scrapy -h
CRAWLER_CLASS
os-scrapy support config CRAWLER_CLASS replace the default scrapy.crawler.Crawler
It can be configured in settings.py file or command line
os-scrapy crawl -c <your_crawler_class> <spider_name>
Enhanced startproject
Thanks to os-scrapy-cookiecutter, -p option can be used to create project as well as python package
os-scrapy startproject -p <project_name> [project_dir]
Set TWISTED_REACTOR on command line
-r option can be used to set TWISTED_REACTOR
os-scrapy crawl -r asyncio <spider_name>
Unit Tests
tox
License
MIT licensed.
1 maintainer
Release files for os-scrapy 0.0.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| os-scrapy-0.0.8.tar.gz | 6.6 kB | Details |
Release files / os-scrapy-0.0.8.tar.gz
| Download URL | os-scrapy-0.0.8.tar.gz |
|---|---|
| Size | 6.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5219bb9ab603489eb10b1e3d1f9910de1c96aaeb7b08de26cfc3cac82de92b9b
|
|
BLAKE2b-256 checksum How to use checksums |
961a878759cbc7b8c9d332df6f20dd0551dc5752c8bbbc000ca49d049c311ff3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.2.0 pkginfo/1.6.1 requests/2.25.0 setuptools/51.0.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.6.7
|
