gunicorn-prometheus-exporter 0.1.4

A comprehensive Prometheus metrics exporter for Gunicorn with support for all worker types (sync, thread, eventlet, gevent, tornado)

Gunicorn Prometheus Exporter

A Gunicorn worker plugin that exports Prometheus metrics to monitor worker performance, including memory usage, CPU usage, request durations, and error tracking (trying to replace https://docs.gunicorn.org/en/stable/instrumentation.html with extra info). It also aims to replace request-level tracking, such as the number of requests made to a particular endpoint, for any framework (e.g., Flask, Django, and others) that conforms to the WSGI specification.

Features

• Worker Metrics: Memory, CPU, request durations, error tracking
• Master Process Intelligence: Signal tracking, restart analytics
• Multiprocess Support: Full Prometheus multiprocess compatibility
• Redis Integration: Forward metrics to Redis for external storage
• Zero Configuration: Works out-of-the-box with minimal setup
• Production Ready: Retry logic, error handling, health monitoring

⚠️ Compatibility Issues

TornadoWorker Compatibility

The PrometheusTornadoWorker has known compatibility issues and is not recommended for production use:

• Metrics Endpoint Hanging: The Prometheus metrics endpoint may hang or become unresponsive
• IOLoop Conflicts: Tornado's event loop architecture conflicts with metrics collection
• Thread Safety Problems: Metrics collection can cause deadlocks

Recommended Alternatives:

• Use PrometheusEventletWorker for async applications requiring eventlet
• Use PrometheusGeventWorker for async applications requiring gevent
• Use PrometheusWorker (sync worker) for most applications

Quick Start

Installation

Basic installation (sync and thread workers only):

pip install gunicorn-prometheus-exporter

With async worker support:

# Install with all async worker types
pip install gunicorn-prometheus-exporter[async]

# Or install specific worker types
pip install gunicorn-prometheus-exporter[eventlet]  # For eventlet workers
pip install gunicorn-prometheus-exporter[gevent]    # For gevent workers
pip install gunicorn-prometheus-exporter[tornado]   # For tornado workers

With Redis forwarding:

pip install gunicorn-prometheus-exporter[redis]

Complete installation (all features):

pip install gunicorn-prometheus-exporter[all]

Basic Usage

Create a Gunicorn config file (gunicorn.conf.py):

# Basic configuration
bind = "0.0.0.0:8000"
workers = 2

# Prometheus exporter (sync worker)
worker_class = "gunicorn_prometheus_exporter.PrometheusWorker"

# Optional: Custom hooks for advanced setup
def when_ready(server):
    from gunicorn_prometheus_exporter.hooks import default_when_ready
    default_when_ready(server)

Supported Worker Types

The exporter supports all major Gunicorn worker types:

Worker Class	Concurrency Model	Use Case	Installation
PrometheusWorker	Pre-fork (sync)	Simple, reliable, 1 request per worker	pip install gunicorn-prometheus-exporter
PrometheusThreadWorker	Threads	I/O-bound apps, better concurrency	pip install gunicorn-prometheus-exporter
PrometheusEventletWorker	Greenlets	Async I/O with eventlet	pip install gunicorn-prometheus-exporter[eventlet]
PrometheusGeventWorker	Greenlets	Async I/O with gevent	pip install gunicorn-prometheus-exporter[gevent]
PrometheusTornadoWorker	Async IOLoop	Tornado-based async (⚠️ Not recommended)	pip install gunicorn-prometheus-exporter[tornado]

Start Gunicorn

gunicorn -c gunicorn.conf.py app:app

Access Metrics

Metrics are automatically exposed on the configured bind address and port (default: 0.0.0.0:9091):

# Using default configuration
curl http://0.0.0.0:9091/metrics

# Or use your configured bind address
curl http://YOUR_BIND_ADDRESS:9091/metrics

Documentation

📖 Complete documentation is available at: https://agent-hellboy.github.io/gunicorn-prometheus-exporter

The documentation includes:

• Installation and configuration guides
• Complete metrics reference
• Framework-specific examples (Django, FastAPI, Flask, Pyramid)
• API reference and troubleshooting
• Contributing guidelines

Available Metrics

Worker Metrics

• gunicorn_worker_requests_total: Total requests processed
• gunicorn_worker_request_duration_seconds: Request duration histogram
• gunicorn_worker_memory_usage_bytes: Memory usage per worker
• gunicorn_worker_cpu_usage_percent: CPU usage per worker
• gunicorn_worker_uptime_seconds: Worker uptime

Master Metrics

• gunicorn_master_signals_total: Signal counts by type
• gunicorn_master_worker_restarts_total: Worker restart counts
• gunicorn_master_workers_current: Current worker count

Redis Metrics (if enabled)

• gunicorn_redis_forwarder_status: Forwarder health status
• gunicorn_redis_forwarder_errors_total: Forwarder error counts

Examples

See the example/ directory for complete working examples with all worker types:

Basic Examples

• gunicorn_simple.conf.py: Basic sync worker setup
• gunicorn_thread_worker.conf.py: Threaded workers for I/O-bound apps
• gunicorn_redis_based.conf.py: Redis forwarding setup

Async Worker Examples

• gunicorn_eventlet_async.conf.py: Eventlet workers with async app
• gunicorn_gevent_async.conf.py: Gevent workers with async app
• gunicorn_tornado_async.conf.py: Tornado workers with async app (⚠️ Not recommended)

Test Applications

• app.py: Simple Flask app for sync/thread workers
• async_app.py: Async-compatible Flask app for async workers

Run any example with:

cd example
gunicorn --config gunicorn_simple.conf.py app:app

Testing Status

All worker types have been thoroughly tested and are production-ready:

Worker Type	Status	Metrics	Master Signals	Load Distribution
Sync Worker	✅ Working	✅ All metrics	✅ HUP, USR1, CHLD	✅ Balanced
Thread Worker	✅ Working	✅ All metrics	✅ HUP, USR1, CHLD	✅ Balanced
Eventlet Worker	✅ Working	✅ All metrics	✅ HUP, USR1, CHLD	✅ Balanced
Gevent Worker	✅ Working	✅ All metrics	✅ HUP, USR1, CHLD	✅ Balanced
Tornado Worker	⚠️ Not recommended	⚠️ Metrics endpoint issues	✅ HUP, USR1, CHLD	✅ Balanced

All async workers require their respective dependencies:

• Eventlet: pip install eventlet
• Gevent: pip install gevent
• Tornado: pip install tornado (⚠️ Not recommended - see compatibility issues)

Configuration

Environment Variables

Variable	Default	Description
PROMETHEUS_METRICS_PORT	9091	Port for metrics endpoint
PROMETHEUS_BIND_ADDRESS	0.0.0.0	Bind address for metrics
GUNICORN_WORKERS	1	Number of workers
PROMETHEUS_MULTIPROC_DIR	Auto-generated	Multiprocess directory
REDIS_ENABLED	false	Enable Redis forwarding
REDIS_URL	redis://127.0.0.1:6379	Redis connection URL (configure for your environment)

Gunicorn Hooks

# Basic setup
from gunicorn_prometheus_exporter.hooks import default_when_ready

def when_ready(server):
    default_when_ready(server)

# With Redis forwarding
from gunicorn_prometheus_exporter.hooks import redis_when_ready

def when_ready(server):
    redis_when_ready(server)

Contributing

Contributions are welcome! Please see our contributing guide for details.

License

This project is licensed under the MIT License - see the LICENSE file for details.