Skip to main content

Export Scrapy spider stats to InfluxDB.

Project description

scrapy-influxdb-exporter

Export Scrapy spider stats to InfluxDB.

This package uses uv for project management. To get started, ensure that uv is installed on your machine and updated to the 0.5.6 version. Detailed installation instructions for uv can be found here.

Installation

uv add scrapy-influxdb-exporter

Usage

Add the following settings to your Scrapy project settings file:

INFLUXDB_DATABASE = ...
INFLUXDB_HOST = ...
INFLUXDB_MEASUREMENT_NAME = ...
INFLUXDB_ORG = ...
INFLUXDB_TOKEN = ...
STATS_CLASS = "scrapy_influxdb_exporter.InfluxDBStatsCollector"

Development

uv sync --group=development
uv run pre-commit install --install-hooks
uv run pre-commit install --hook-type=commit-msg

Acknowledgments

This project was inspired by scrapy-prometheus-exporter by @rangertaha and this article by @mikulskibartosz.

Contributing

Contributions are welcome! To get started, please refer to our contribution guidelines.

Issues

If you encounter any problems while using this package, please open a new issue here.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy_influxdb_exporter-1.3.0.tar.gz (33.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

scrapy_influxdb_exporter-1.3.0-py3-none-any.whl (4.2 kB view details)

Uploaded Python 3

File details

Details for the file scrapy_influxdb_exporter-1.3.0.tar.gz.

File metadata

File hashes

Hashes for scrapy_influxdb_exporter-1.3.0.tar.gz
Algorithm Hash digest
SHA256 7b23e7dfbcc920c9487d7280de3cf0012449ed5d925389f32cb154c4aa4e5c07
MD5 697bec9a7ea0eb1b4c407d719e304ea0
BLAKE2b-256 bf1c111cd9a738ec9d58873e280f07fa0aed5f87d6d51074f36a806ffb4ccc01

See more details on using hashes here.

File details

Details for the file scrapy_influxdb_exporter-1.3.0-py3-none-any.whl.

File metadata

File hashes

Hashes for scrapy_influxdb_exporter-1.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d7265e45880be5f3e1418ff6e365c07b491c884d00ce1bc3578b7df4b2a4cee4
MD5 a900551e39ea6e3ada366f2ab22e8fc5
BLAKE2b-256 3aaa1f1fbece20891122b8f7db406b0ec48ca4e9aea18c585db6aaa4ba5c7c60

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page