用于爬取JD和github的代码，数据存储于mongodb。.
Project description
scrapy crawl test -s JOBDIR=data -0 items.csv
watch -n 1 -d 'wc -l items.csv;du -h items.csv'
附数据 items.csv
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapy_jingdong-1.2.1.tar.gz
(4.7 kB
view details)
Built Distribution
File details
Details for the file scrapy_jingdong-1.2.1.tar.gz.
File metadata
- Download URL: scrapy_jingdong-1.2.1.tar.gz
- Upload date:
- Size: 4.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 2a7ed8cfa9ba258da96b08d324ab2bd7785f8ea16f766909c6609ff29231d14a |
|
| MD5 | e332b9154cfb95eb01cf3bc324de3bbd |
|
| BLAKE2b-256 | 0730c90010a7118bb9a6771a715b4e2f2015d15fd52352d232ba13d7eebbc3a4 |
File details
Details for the file scrapy_jingdong-1.2.1-py3-none-any.whl.
File metadata
- Download URL: scrapy_jingdong-1.2.1-py3-none-any.whl
- Upload date:
- Size: 5.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.11.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ee760df7aead5859d98da93207af683403f73d69d29d804ca4ba6be20e16533e |
|
| MD5 | afdec8a25c7fe875a5f25e29b97edb86 |
|
| BLAKE2b-256 | 525a980847cfc269448a66f59db89781f7cda4d4dbdab7201db4f95c8e0249bc |
