os-scrapy
Ozzy's Scrapy scaffold.
Requirements
- Python 3.6+ (pypy3.6+)
- Scrapy 2.0+
Install
pip install os-scrapy
Usage
Command line
The command is same as scrapy
os-scrapy -h
CRAWLER_CLASS
os-scrapy support config CRAWLER_CLASS replace the default scrapy.crawler.Crawler
It can be configured in settings.py file or command line
os-scrapy crawl -c <your_crawler_class> <spider_name>
Enhanced startproject
Thanks to os-scrapy-cookiecutter, -p option can be used to create project as well as python package
os-scrapy startproject -p <project_name> [project_dir]
Set TWISTED_REACTOR on command line
-r option can be used to set TWISTED_REACTOR
os-scrapy crawl -r asyncio <spider_name>
Unit Tests
tox
License
MIT licensed.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
os-scrapy-0.0.8.tar.gz
(6.6 kB
view details)
File details
Details for the file os-scrapy-0.0.8.tar.gz.
File metadata
- Download URL: os-scrapy-0.0.8.tar.gz
- Upload date:
- Size: 6.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/3.2.0 pkginfo/1.6.1 requests/2.25.0 setuptools/51.0.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.6.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
5219bb9ab603489eb10b1e3d1f9910de1c96aaeb7b08de26cfc3cac82de92b9b
|
|
| MD5 |
4e17c4d0f0ede929ee6816787b2ea30d
|
|
| BLAKE2b-256 |
961a878759cbc7b8c9d332df6f20dd0551dc5752c8bbbc000ca49d049c311ff3
|
