Scrapy extension that logs errors to Sentry
Project description
🐞 scrapy-sentry-errors
A simple Scrapy extension that logs spider errors to your Sentry account, helping you monitor and fix issues with your Scrapy spiders efficiently. 🚀
Note: While this extension captures errors from Scrapy spiders, it does not capture errors elsewhere in Scrapy's operation (e.g., the Scrapy pipeline).
📋 Requirements
🛠️ Installation
pip install city-scrapers-sentry
🏗️ Setup
Add SENTRY_DSN
and scrapy_sentry_errors.extensions.Errors
extension to your Scrapy project's settings.py
file:
Example:
# sentry dsn
SENTRY_DSN = 'http://public:secret@example.com/1'
EXTENSIONS = {
"scrapy_sentry_errors.extensions.Errors": 10,
}
Sentry spider errors will be logged to your Sentry account.
💻 Development
-
Clone this repository.
-
Run the dev setup script and follow the prompts. This will create a virtual environment and a
.env
file with your Sentry DSN:
bash ./scripts/dev_setup.sh
- Simulate a failed spider run using the example Scrapy project in this repo:
pipenv run example
🧪 Testing
Run the tests with:
pipenv run test
Deployment
-
Bump the version number in
pyproject.toml
-
Create a new build with:
pipenv run build
- Publish the build to PyPI with:
pipenv run deploy
🤝 Contributing
We welcome contributions of all kinds from anyone. Here are some ways you can contribute:
- Reporting Bugs: If you find a bug, please report it by opening an issue. Be sure to provide a detailed description of the bug and, if possible, steps to reproduce it.
- Suggesting Enhancements: Have an idea for a new feature or think something could be improved? Let us know by opening an issue.
- Code Contributions: Feel free to fork the repository and submit pull requests. Whether it's fixing a bug, adding a feature, or improving documentation, your help is welcome.
🙏 Acknowledgements
This project began as a fork of scrapy-sentry, which was developed by Jordi Llonch. We are grateful for his work and those of other contributors.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for scrapy-sentry-errors-1.0.0b2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | cbf4efa1b6481ac813322ba1ba0ea43f7469a34f02460b2a0abe4c13eab7b879 |
|
MD5 | ebd17bc4e1cfd60e822e347f45614a7a |
|
BLAKE2b-256 | ac8b5668a9193e03bb012d97ebae3c84b9182d975b8423759d6bd423b7255648 |
Hashes for scrapy_sentry_errors-1.0.0b2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ad989ba9a0f7d9913281a977881f9aa8fbbe4f42409d5971187a28a9056fa8fa |
|
MD5 | 315a47f3483c16a3df8f7e42b3b72d3d |
|
BLAKE2b-256 | aebe9db089b3e1e3b01cdb8019ce7c3d95ff4efb05c4e1a631fb64942a750ef3 |