Ozzy's Scrapy
Project description
os-scrapy
Ozzy's Scrapy scaffold.
Requirements
- Python 3.6+ (pypy3.6+)
- Scrapy 2.0+
Install
pip install os-scrapy
Usage
Command line
The command is same as scrapy
os-scrapy -h
CRAWLER_CLASS
os-scrapy support config CRAWLER_CLASS
replace the default scrapy.crawler.Crawler
It can be configured in settings.py
file or command line
os-scrapy crawl -c <your_crawler_class> <spider_name>
Enhanced startproject
Thanks to os-scrapy-cookiecutter, -p
option can be used to create project as well as python package
os-scrapy startproject -p <project_name> [project_dir]
Set TWISTED_REACTOR
on command line
-r
option can be used to set TWISTED_REACTOR
os-scrapy crawl -r asyncio <spider_name>
Unit Tests
tox
License
MIT licensed.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
os-scrapy-0.0.8.tar.gz
(6.6 kB
view hashes)