Skip to main content

Helpers and examples to build Scrapy Crawlers in a test driven way.

Project description

scrapy_tdd

Helpers and examples to build Scrapy Crawlers in a test driven way.

Motivation

… coming soon …

Why not scrapy’s contracts?

… coming soon …

Installation

pip install scrapy_tdd

Quick Start Examples

def describe_fancy_spider():

to_test = MySpider().from_crawler(get_crawler())

def describe_parse_suggested_terms():

resp = response_from(“Result_JSON_Widget.txt”) results = to_test.parse(resp)

def should_get_item():

item = results assert item[0][“lorem”] == ‘ipsum’ assert item[0][“iterem”] == “ipsem”

Full Documentation

… coming soon …

Missing / next steps

  • Python 3.x compatibility

  • Mocking Request-Response pairs

How to contribute

… coming soon …

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy_tdd-0.1.2rc1.tar.gz (3.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page