executor-exporter 0.1.1

No project description provided

executor-exporter

A Prometheus exporter for Python concurrent.futures executors. Provides instrumented drop-in replacements for ThreadedPoolExecutor and ProcessPoolExecutor.

Public domain photo by Patsy Lynch. More info

Install it from PyPI

pip install executor-exporter

Usage

from executor_exporter import ThreadPoolExecutor
# or
from executor_exporter import ProcessPoolExecutor

If you stick to the public APIs of concurrent.future executors (consisting of __init__, submit and map methods), you just need to replace the builtin executor with its instrumented version provided by this package.

The provided executors act as proxies for the builtin executor while collecting the following metrics:

Additional parameters

The __init__ methods of the instrumented executors take two additional optional parameters:

Parameter	Type	Default value	Description
exporter_id	str	""	This id is used as the value for the exporter label in all metrics. Useful when your app uses multiple executors and you want to measure them separately
registry	prometheus_client.CollectorRegistry	prometheus_client.REGISTRY (default registry)	Useful when you're using a registry other than the default for whatever reason, e.g. using prometheus_client multiprocess mode

Custom executors

The InstrumentedExecutorProxy class does the heavy-lifting. If you're using a custom executor, you can still instrument them by using wrapping it:

from executor_exporter import InstrumentedExecutorProxy, ExecutorExporter

max_workers = 42
executor = YourCustomExecutor(max_workers)
exporter = ExecutorExporter(executor)

instrumented_executor = InstrumentedExecutorProxy(executor, exporter, max_workers)