Scrapy extension that logs errors to Sentry
Project description
city-scrapers-sentry
A Scrapy extension that logs spider errors to Sentry.
This project began as a fork of scrapy-sentry, which was developed by Jordi Llonch. We are grateful for his work and those of other contributors.
Note: while this extension captures errors from Scrapy spiders, it does not capture errors elsewhere in Scrapy's operation (e.g. the Scrapy pipeline).
Requirements:
Installation
pip install city-scrapers-sentry
Setup
Add SENTRY_DSN
and city_scrapers_sentry.extensions.Errors
extension to your Scrapy project's settings.py
file:
Example:
# sentry dsn
SENTRY_DSN = 'http://public:secret@example.com/1'
EXTENSIONS = {
"city_scrapers_sentry.extensions.Errors": 10,
}
Development
-
Install pipenv if you don't have it already.
-
At the project root, create a .env file and include the following. Replace
<your-sentry-dsn>
with your Sentry DSN:
SENTRY_DSN=<your-sentry-dsn>
- Run the following command from the project root to activate pipenv's virtual environment and install project dependencies:
pipenv sync --dev
When activated, pipenv should load your SENTRY_DSN env var from the .env file.
- Enter the example_project directory:
cd example_project
- Simulate a failed spider run. The spider will log an error to your Sentry account:
scrapy crawl example
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for city-scrapers-sentry-1.0.0a4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 736e43ee6ad764b31f72611fb5f52d6ea3c45de0e0994acf529e1ca775db2a53 |
|
MD5 | 1610ee9900ca1538d7d880d080932d2b |
|
BLAKE2b-256 | d9941bdc0560b71e4d41e8ce20752ca4ee8f01a619514cb9533904f077ddfed1 |
Hashes for city_scrapers_sentry-1.0.0a4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b69176ea9af56e5a961c243b21d334d0068818011fb4c34e431a2215b7dadcef |
|
MD5 | ff1eb7562dec63055abe99816059a17a |
|
BLAKE2b-256 | 49c2d7bdc8a7d11bef6e755fea78f5a654defe17b3e9393e7536d4369e72ed53 |