Skip to main content

Package for buildout based scrapy spider development

Project description

This package provides some core component for buildout based scrapy spider
development. Such scrapy spider packages can get installed, scheduled and
processed with the mongodb based s01.worker daemon using the JSON-RPC proxy
located in the s01.client package. The package also provides some recipes
which allows to use external files as scrapy settings.


=======
Changes
=======

0.12.0 (2011-08-19)
------------------

- nail scrapy release version to 0.12.0.2546

- removed unused dependencies and imports

- initial release


0.0.7 (2011-01-02)
------------------

- alpha version released for development and testing the tool chain

- write logging.Error to sys.stderr where we can read in subprocess


0.0.6 (2010-12-31)
------------------

- alpha version released for development and testing the tool chain

- print logging.ERROR to stdout which is required for error handling in
subprocess


0.0.5 (2010-12-29)
------------------

- alpha version released for development and testing the tool chain

- implemented a different scrapy item and field concept. Use a field property
and not a dict based item and field. Implemented ScrapyFieldProperty and a
ScrapyItemBase class. Added tests for show how the scrapy item and field
works including converter and serializer.

- implemented new extractor which can handle the new scrapy item and field
concept

- implemented different basic ScrapyFieldProperty converter methods


0.0.4 (2010-12-22)
------------------

- alpha version released for development and testing the tool chain

- remove spider name from crawl recipe


0.0.3 (2010-11-29)
------------------

- alpha version released for development and testing the tool chain

- fix hex data parts in settings content

- use s01.worker as default logging handler name


0.0.2 (2010-11-29)
------------------

- alpha version released for development and testing the tool chain

- added settings recipe


0.0.1 (2010-11-21)
------------------

- alpha version released for development and testing the tool chain

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

s01.scrapy-0.12.0.zip (28.2 kB view details)

Uploaded Source

File details

Details for the file s01.scrapy-0.12.0.zip.

File metadata

  • Download URL: s01.scrapy-0.12.0.zip
  • Upload date:
  • Size: 28.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for s01.scrapy-0.12.0.zip
Algorithm Hash digest
SHA256 924d16ce6bf8f3f488956efea54870ce48f5f0a8035e377990fa560a1412b76b
MD5 71dd3a4df67b6a2495a9a57df7c3304f
BLAKE2b-256 c7c6355fd2aa8697af0a07781f9fad4d71ff64533c5f25ae60f54e8816587072

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page