django-mailer 2.1

A reusable Django app for queuing the sending of email

First, add “mailer” to your INSTALLED_APPS in your settings.py. Run ./manage.py migrate to install models.

Using EMAIL_BACKEND

This is the preferred and easiest way to use django-mailer.

To automatically switch all your mail to use django-mailer, first set EMAIL_BACKEND:

EMAIL_BACKEND = "mailer.backend.DbBackend"

If you were previously using a non-default EMAIL_BACKEND, you need to configure the MAILER_EMAIL_BACKEND setting, so that django-mailer knows how to actually send the mail:

MAILER_EMAIL_BACKEND = "your.actual.EmailBackend"

Now, just use the normal Django mail functions for sending email. These functions will store mail on a queue in the database, which must be sent as below.

Explicitly putting mail on the queue

If you don’t want to send all email through django-mailer, you can send mail using mailer.send_mail, which has the same signature as Django’s send_mail function.

You can also do the following:

# favour django-mailer but fall back to django.core.mail
from django.conf import settings

if "mailer" in settings.INSTALLED_APPS:
    from mailer import send_mail
else:
    from django.core.mail import send_mail

and then just call send_mail like you normally would in Django:

send_mail(subject, message_body, settings.DEFAULT_FROM_EMAIL, recipients)

There is also a convenience function mailer.send_html_mail for creating HTML (this function is not in Django):

send_html_mail(subject, message_plaintext, message_html, settings.DEFAULT_FROM_EMAIL, recipients)

Additionally you can send all the admins as specified in the ADMIN setting by calling:

mail_admins(subject, message_body)

or all managers as defined in the MANAGERS setting by calling:

mail_managers(subject, message_body)

Clear queue with command extensions

With mailer in your INSTALLED_APPS, there will be three new manage.py commands you can run:

• send_mail will clear the current message queue. If there are any failures, they will be marked deferred and will not be attempted again by send_mail.

• retry_deferred will move any deferred mail back into the normal queue (so it will be attempted again on the next send_mail).

• purge_mail_log will remove old successful message logs from the database, to prevent it from filling up your database. Use the -r failure option to remove only failed message logs instead, or -r all to remove them all.

You may want to set these up via cron to run regularly:

*       * * * * (/path/to/your/python /path/to/your/manage.py send_mail >> ~/cron_mail.log 2>&1)
0,20,40 * * * * (/path/to/your/python /path/to/your/manage.py retry_deferred >> ~/cron_mail_deferred.log 2>&1)
0 0 * * * (/path/to/your/python /path/to/your/manage.py purge_mail_log 7 >> ~/cron_mail_purge.log 2>&1)

For use in Pinax, for example, that might look like:

* * * * * (cd $PINAX; /usr/local/bin/python2.5 manage.py send_mail >> $PINAX/cron_mail.log 2>&1)
0,20,40 * * * * (cd $PINAX; /usr/local/bin/python2.5 manage.py retry_deferred >> $PINAX/cron_mail_deferred.log 2>&1)
0 0 * * * (cd $PINAX; /usr/local/bin/python2.5 manage.py purge_mail_log 7 >> $PINAX/cron_mail_purge.log 2>&1)

This attempts to send mail every minute with a retry on failure every 20 minutes, and purges the mail log for entries older than 7 days.

manage.py send_mail uses a lock file in case clearing the queue takes longer than the interval between calling manage.py send_mail.

Note that if your project lives inside a virtualenv, you also have to execute this command from the virtualenv. The same, naturally, applies also if you’re executing it with cron. The Pinax documentation explains that in more details.

Controlling the delivery process

If you wish to have a finer control over the delivery process, which defaults to deliver everything in the queue, you can use the following 3 variables (default values shown):

MAILER_EMAIL_MAX_BATCH = None  # integer or None
MAILER_EMAIL_MAX_DEFERRED = None  # integer or None
MAILER_EMAIL_THROTTLE = 0  # passed to time.sleep()

These control how many emails are sent successfully before stopping the current run MAILER_EMAIL_MAX_BATCH, after how many failed/deferred emails should it stop MAILER_EMAIL_MAX_DEFERRED and how much time to wait between each email MAILER_EMAIL_THROTTLE.

Unprocessed emails will be evaluated in the following delivery iterations.

Error handling

django-mailer comes with a default error handler mailer.engine.handle_delivery_exception.

It marks the related message as deferred for any of these exceptions:

• smtplib.SMTPAuthenticationError

• smtplib.SMTPDataError

• smtplib.SMTPRecipientsRefused

• smtplib.SMTPSenderRefused

• socket.error

Any other exceptions is re-raised. That is done for backwords-compatiblity as well as for flexibility: we would otherwise have to maintain an extensive and changing list of exception types, which does not scale, and you get the chance to do error handling that fits your environment like a glove.

When the default behavior does not fit your environment, you can specify your own custom delivery error handler through setting MAILER_ERROR_HANDLER. The value should be a string for use with Django’s import_string, the default is "mailer.engine.handle_delivery_exception".

Your handler is passed three arguments, in order:

• connection — the backend connection instance that failed delivery

• message — the Message instance that failed delivery

• exc — the exception instance raised by the mailer backend

Your handler should return a 2-tuple of:

1. a connection instance (or None to cause a new connection to be created)

2. a string denoting the action taken by the handler, either "sent" or "deferred" precisely

For an example of a custom error handler:

def my_handler(connection, message, exc):
    if isinstance(exc, SomeDeliveryException):
        # trying to re-send this very message desparately
        # (if you have good reason to)
        [..]
        status = 'sent'
    elif isinstance(exc, SomeOtherException):
        message.defer()
        connection = None  # i.e. ask for a new connection
        status = 'deferred'
    else:
        six.reraise(*sys.exc_info())

    return connection, status

Other settings

If you need to be able to control where django-mailer puts its lock file (used to ensure mail is not sent twice), you can set MAILER_LOCK_PATH to a full absolute path to the file to be used as a lock. The extension “.lock” will be added. The process running send_mail needs to have permissions to create and delete this file, and others in the same directory. With the default value of None django-mailer will use a path in current working directory.

If you need to disable the file-based locking, you can set the MAILER_USE_FILE_LOCK setting to False.

If you need to change the batch size used by django-mailer to save messages in mailer.backend.DbBackend, you can set MAILER_MESSAGES_BATCH_SIZE to a value more suitable for you. This value, which defaults to None, will be passed to Django’s bulk_create method as the batch_size parameter.

Using the DontSendEntry table

django-mailer creates a DontSendEntry model, which is used to filter out recipients from messages being created.

But beware, it’s actually only used when directly sending messages through mailer, not when mailer is used as an alternate EMAIL_BACKEND for Django. Also, even if recipients become empty due to this filtering, the email will be queued for sending anyway. (A patch to fix these issues would be accepted) Change log ==========

2.1 - 2020-12-05

• The retry_deferred and send_mail commands rely on the log level set in your django project now. The -c/--cron option in those commands has been deprecated and the logic to configure log levels and the message format has been removed.

• Changed logging to use module specific loggers to avoid interfering with other loggers.

• Added MAILER_USE_FILE_LOCK setting to allow disabling file based locking.

• Added -r option to purge_mail_log management command. Thanks julienc91

• Fixed deprecation warnings on Django 3.1

• Use cached DNS_NAME for performance

• Added ability to override the default error handler via the MAILER_ERROR_HANDLER settings key

2.0.1 - 2020-03-01

• Fixed issue with migration that some people experienced (see PR 118)

2.0 - 2019-09-23

• Django 3.0 support

• Dropped support for old Django versions (before 1.11)

• Changed DB priority field to an integer, instead of text field container an integer

• Multi-process safety for sending emails via database row-level locking.

  Previously, there was a file-system based lock to ensure that multiple processes were not attempting to send the mail queue, to stop multiple sending of the same email. However, this mechanism only works if all processes that might be attempting to do this are on the same machine with access to the same file-system.

  Now, in addition to this file lock, we use transactions and row-level locking in the database when attempting to send a message, which guarantees that only one process can send the message. In addition, for databases that support NOWAIT with SELECT FOR UPDATE, such as PostgreSQL, if multiple processes attempt to send the mail queue at the same time, the work should be distributed between them (rather than being done by only one process).

  A negative consequence is that SQLite support is degraded: due to the way it implements locking and our use of transactions when sending the email queue, you can get exceptions in other processes that are trying to add items to the queue. Use of SQLite with django-mailer is not recommended.

• retry_deferred command has also been updated to be simpler and work correctly for multiple processes.

• Dropped some backwards compat support for Django < 1.8. If you are upgrading from a version of Django before 1.8, you should install a version of django-mailer < 2.0, do send_all to flush the queue, then upgrade django-mailer to 2.0 or later.

1.2.6 - 2019-04-03

• Official Django 2.1 and 2.2 support.

• Don’t close DB connection in management commands. This is unnecessary with modern Django.

1.2.5

• Fixed packaging file permission problems.

• Added Japanese locale (thanks msk7777)

1.2.4

• Django 2.0 support.

1.2.3

• Fixed crasher with models __str__

1.2.2

• Django 1.10 support.

• Fixed reprs for Message and MessageLog.

1.2.1

• More helpful admin for Message and MessageLog

• Handle exceptions from really old Django versions

1.2.0

• Save the Message-ID header on Message explicitly to enable finding emails using this identifier.

  This includes a database schema migration.

1.1.0

• Deprecated calling send_mail and send_html_mail using priority kwargs "high", "medium", and "low". Instead you should use PRIORITY_HIGH, PRIORITY_MEDIUM and PRIORITY_LOW from mailer.models.

• Fixed bug with migrations for Django 1.7, which wanted to create a migration to ‘fix’ the EmailField length back down to 75 instead of 254.

1.0.1

• Included migrations - for both South and Django 1.7 native migrations.

  Note:

  • If you use South, you will need at least South 1.0

  • You will need to use ‘–fake’ or ‘–fake-initial’ on existing installations.

  These migrations were supposed to be in 1.0.0 but were omitted due to a packaging error.

1.0.0

• Throttling of email sending

• Django 1.8 support

• Admin tweaks and improvements

• Various other fixes, especially from Renato Alves <[e-mail]> - thank you!

0.1.0

• First PyPI version