Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Scrapy pipeline extension for qiniu.com

Project Description
scrapy_qiniu
=====

Scrapy中的media pipeline机制,可以方便地将静态资源资源(文件、图像等)下载到本地,然后进行处理。scrapy-qiniu扩展了这一机制,可以将资源存储到七牛云存储上面。并且,实现了以下几个特性:

* 支持缓存,可以避免静态资源的重复下载
* 采用fetch模式,让七牛服务器代为下载,而不用像默认的FilesPipeline那样,先下载到爬虫所在
主机,然后再上传到七牛服务器
Release History

Release History

This version
History Node

0.1.2

History Node

0.1.1

History Node

0.1.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
scrapy_qiniu-0.1.2-py2-none-any.whl (5.7 kB) Copy SHA256 Checksum SHA256 py2 Wheel Dec 26, 2015
scrapy_qiniu-0.1.2.tar.gz (3.9 kB) Copy SHA256 Checksum SHA256 Source Dec 26, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting