SMTPHandler Example

A typical use case may be to throttle error notification emails sent by the logging.handlers.SMTPHandler.

Here’s an example of how you might set that up:

import logging.handlers
import time

from ratelimitingfilter import RateLimitingFilter

logger = logging.getLogger('throttled_smtp_example')

# Create an SMTPHandler
smtp = logging.handlers.SMTPHandler(
    mailhost='smtp.example.com',
    fromaddr='from@example.com',
    toaddrs='to@example.com',
    subject='An error has occurred'
)
smtp.setLevel(logging.ERROR)

# Create a formatter and set it on the handler
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
smtp.setFormatter(formatter)

# Create an instance of the RateLimitingFilter, and add it to the handler
ratelimit = RateLimitingFilter()
smtp.addFilter(ratelimit)

# Add the handler to the logger
logger.addHandler(smtp)

# Logged errors will now be restricted to 1 every 30 seconds
while True:
    logger.error('An error message')
    time.sleep(2)

Substring based message throttling

You can pass a list of substrings to the RateLimitingFilter which it will use to match messages to apply to.

config = {'match': ['some error', 'a different error']}

ratelimit = RateLimitingFilter(rate=1, per=60, burst=1, **config)
smtp.addFilter(ratelimit)

# Can be rate limited
logger.error('some error occurred')

# Can be rate limited
logger.error('a different error occurred')

# Will not be rate limited
logger.error('something completely different happened')

Automatic message throttling

This is an experimental feature.

You can let the RateLimitingFilter automatically throttle messages by setting the match option to auto.

config = {'match': 'auto'}
ratelimit = RateLimitingFilter(rate=1, per=60, burst=1, **config)

The filter will then attempt to identify messages based on their content in order to figure out whether to throttle them or not. It will tolerate slight differences in content when identifying messages. So for example, if error messages are being rapidly logged that are the same apart from a timestamp, or perhaps an incrementing id, then these messages will be treated as the same as far as rate limiting is concerned.