High-performance async HTTP logging handler for Datadog with batching and retry logic
Project description
Datadog Async Handler
A modern, high-performance Python logging handler that sends logs directly to Datadog via HTTP API with asynchronous batching, retry logic, and comprehensive error handling.
โจ Features
- ๐ High Performance: Asynchronous batching and background processing
- ๐ Reliable Delivery: Automatic retry with exponential backoff
- ๐ Batching: Configurable batch size and flush intervals
- ๐ท๏ธ Rich Metadata: Automatic service, environment, and custom tag support
- ๐ง Easy Integration: Drop-in replacement for standard logging handlers
- ๐ Multi-Site Support: Works with all Datadog sites (US, EU, etc.)
- ๐ Type Safe: Full type hints and mypy compatibility
- โก Modern: Built with Python 3.9+ and latest best practices
๐ Quick Start
Installation
pip install datadog-async-handler
Basic Usage
import logging
from datadog_http_handler import DatadogHTTPHandler
# Configure the handler
handler = DatadogHTTPHandler(
api_key="your-datadog-api-key", # or set DD_API_KEY env var
service="my-application",
source="python",
tags="env:production,team:backend"
)
# Set up logging
logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.INFO)
# Start logging!
logger.info("Application started successfully", extra={
"user_id": "12345",
"action": "startup"
})
Environment Variables
The handler automatically picks up standard Datadog environment variables:
export DD_API_KEY="your-api-key"
export DD_SERVICE="my-application"
export DD_ENV="production"
export DD_VERSION="1.2.3"
export DD_TAGS="team:backend,component:api"
export DD_SITE="datadoghq.com" # or datadoghq.eu, ddog-gov.com, etc.
๐ Documentation
Configuration Options
| Parameter | Type | Default | Description |
|---|---|---|---|
api_key |
str |
None |
Datadog API key (required) |
site |
str |
"datadoghq.com" |
Datadog site |
service |
str |
None |
Service name |
source |
str |
"python" |
Log source |
hostname |
str |
None |
Hostname |
tags |
str |
None |
Comma-separated tags |
batch_size |
int |
10 |
Number of logs per batch |
flush_interval_seconds |
float |
5.0 |
Batch flush interval |
timeout_seconds |
float |
10.0 |
Request timeout |
max_retries |
int |
3 |
Maximum retry attempts |
Framework Integration Examples
Django
# settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'datadog': {
'class': 'datadog_http_handler.DatadogHTTPHandler',
'api_key': 'your-api-key',
'service': 'django-app',
'source': 'django',
},
},
'root': {
'handlers': ['datadog'],
'level': 'INFO',
},
}
FastAPI
import logging
from fastapi import FastAPI
from datadog_http_handler import DatadogHTTPHandler
app = FastAPI()
# Configure logging
handler = DatadogHTTPHandler(service="fastapi-app", source="fastapi")
logging.getLogger().addHandler(handler)
logging.getLogger().setLevel(logging.INFO)
@app.get("/")
async def root():
logging.info("API endpoint called", extra={"endpoint": "/"})
return {"message": "Hello World"}
Flask
import logging
from flask import Flask
from datadog_http_handler import DatadogHTTPHandler
app = Flask(__name__)
# Configure logging
handler = DatadogHTTPHandler(service="flask-app", source="flask")
app.logger.addHandler(handler)
app.logger.setLevel(logging.INFO)
@app.route("/")
def hello():
app.logger.info("Flask endpoint called", extra={"endpoint": "/"})
return "Hello World!"
๐ง Development
Setup
# Clone the repository
git clone https://github.com/enlyft/datadog-http-handler.git
cd datadog-http-handler
# Install UV (modern Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Install dependencies
uv pip install -e ".[dev]"
# Install pre-commit hooks
pre-commit install
Running Tests
# Run all tests
hatch run test
# Run with coverage
hatch run test-cov
# Run specific tests
hatch run test tests/test_handler.py::test_basic_logging
Code Quality
# Format code
hatch run format
# Lint code
hatch run lint
# Type checking
hatch run type-check
# Run all checks
hatch run all
๐ค Contributing
We welcome contributions! Please see our Contributing Guide for details.
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests
- Run the test suite
- Submit a pull request
๐ Requirements
- Python 3.9+
datadog-api-client>=2.0.0
๐ License
This project is licensed under the MIT License - see the LICENSE file for details.
๐ Links
๐ Comparison with Other Solutions
| Feature | datadog-async-handler | datadog-http-handler | python-datadog | datadog-logger |
|---|---|---|---|---|
| Async Batching | โ | โ | โ | โ |
| Retry Logic | โ | โ | โ | โ |
| Type Hints | โ | โ | โ | โ |
| Modern Python | โ (3.9+) | โ (3.6+) | โ (2.7+) | โ (3.6+) |
| Official API Client | โ | โ | โ | โ |
| Background Processing | โ | โ | โ | โ |
| Memory Efficient | โ | โ | โ | โ |
| Active Maintenance | โ | โ (2019) | โ | โ |
| Comprehensive Tests | โ | โ | โ | โ |
Made with โค๏ธ for the Python and Datadog communities.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file datadog_async_handler-0.1.3.tar.gz.
File metadata
- Download URL: datadog_async_handler-0.1.3.tar.gz
- Upload date:
- Size: 119.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3e849d47656ce7eff9775a2d9074f6a9b7d5a6db462dc787493bfa06937726d8
|
|
| MD5 |
c002f264e9bf0a2e66ace2fb50e731f2
|
|
| BLAKE2b-256 |
75203064f33fb6cf0e12c9c4a0840a5bcda7deb8608c2579d14210b069034978
|
Provenance
The following attestation bundles were made for datadog_async_handler-0.1.3.tar.gz:
Publisher:
publish.yml on enlyft/datadog-async-handler
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
datadog_async_handler-0.1.3.tar.gz -
Subject digest:
3e849d47656ce7eff9775a2d9074f6a9b7d5a6db462dc787493bfa06937726d8 - Sigstore transparency entry: 597679297
- Sigstore integration time:
-
Permalink:
enlyft/datadog-async-handler@bde4bce2ea3847b5a3d087231ab9d7ccec0cad3c -
Branch / Tag:
refs/tags/v0.1.3 - Owner: https://github.com/enlyft
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@bde4bce2ea3847b5a3d087231ab9d7ccec0cad3c -
Trigger Event:
release
-
Statement type:
File details
Details for the file datadog_async_handler-0.1.3-py3-none-any.whl.
File metadata
- Download URL: datadog_async_handler-0.1.3-py3-none-any.whl
- Upload date:
- Size: 10.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2d044d067136f3b2440780566eeb69fe98d7427ffaad22de3e7c1e7a8b969f67
|
|
| MD5 |
97dc1da3180f5de9de6df5602095432b
|
|
| BLAKE2b-256 |
4ce06a06b7531e59cb7896d66f7551a3b7a97a8b8c822d284c6a950bc30c570d
|
Provenance
The following attestation bundles were made for datadog_async_handler-0.1.3-py3-none-any.whl:
Publisher:
publish.yml on enlyft/datadog-async-handler
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
datadog_async_handler-0.1.3-py3-none-any.whl -
Subject digest:
2d044d067136f3b2440780566eeb69fe98d7427ffaad22de3e7c1e7a8b969f67 - Sigstore transparency entry: 597679303
- Sigstore integration time:
-
Permalink:
enlyft/datadog-async-handler@bde4bce2ea3847b5a3d087231ab9d7ccec0cad3c -
Branch / Tag:
refs/tags/v0.1.3 - Owner: https://github.com/enlyft
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@bde4bce2ea3847b5a3d087231ab9d7ccec0cad3c -
Trigger Event:
release
-
Statement type: