Skip to main content

Export Scrapy spider stats to InfluxDB.

Project description

scrapy-influxdb-exporter

Export Scrapy spider stats to InfluxDB.

This package uses uv for project management. To get started, ensure that uv is installed on your machine and updated to the 0.5.6 version. Detailed installation instructions for uv can be found here.

Installation

uv add scrapy-influxdb-exporter

Usage

Add the following settings to your Scrapy project settings file:

INFLUXDB_DATABASE = ...
INFLUXDB_HOST = ...
INFLUXDB_MEASUREMENT_NAME = ...
INFLUXDB_ORG = ...
INFLUXDB_TOKEN = ...
STATS_CLASS = "scrapy_influxdb_exporter.InfluxDBStatsCollector"

Development

uv sync --frozen --group=development
uv run --frozen pre-commit install --install-hooks
uv run --frozen pre-commit install --hook-type=commit-msg

Acknowledgments

This project was inspired by scrapy-prometheus-exporter by @rangertaha and this article by @mikulskibartosz.

Contributing

Contributions are welcome! To get started, please refer to our contribution guidelines.

Issues

If you encounter any problems while using this package, please open a new issue here.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy_influxdb_exporter-1.4.0.tar.gz (33.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

scrapy_influxdb_exporter-1.4.0-py3-none-any.whl (4.3 kB view details)

Uploaded Python 3

File details

Details for the file scrapy_influxdb_exporter-1.4.0.tar.gz.

File metadata

File hashes

Hashes for scrapy_influxdb_exporter-1.4.0.tar.gz
Algorithm Hash digest
SHA256 1c40628d91c1ca3f6b90911f7d157aa62a9d025fca725674fee967e6fb34d104
MD5 1bcf158415c9d430aa739b9db1f1a6b0
BLAKE2b-256 8d9454d6665129fa43ddb792e9360bf8996c304e2339b8859a115aff3e766094

See more details on using hashes here.

File details

Details for the file scrapy_influxdb_exporter-1.4.0-py3-none-any.whl.

File metadata

File hashes

Hashes for scrapy_influxdb_exporter-1.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 785ad35077fb0b5e3746528d5cd7ede49e1cfcd39b0da6607d3560f31061cf5f
MD5 173489a6654f04a2cb4c66af838092fc
BLAKE2b-256 ac419b81a4f6e3cbfe1a7b8201da7f73053898da44a59d6640216d0e12d218db

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page