django-inbound-email 0.13

A Django app for receiving inbound emails.

Django Inbound Email

This package now supports Python3 and Django1.11 or above. For previous version support please refer to the Python2 branch.

An inbound email handler for Django.

Current Status

We have a working implementation, with SendGrid, Mailgun and Mandrill backends. (Both SendGrid and Mandrill have been used in production environments.)

The test_app is deployed to Heroku and any emails sent to [e-mail] will be parsed by a live version of the test_app.

This test app just does a bounceback - any incoming email to that address is sent back to the sender’s address. This makes it easy to test ‘real-world’ email examples. Please feel free to send email to that address - and if you’re using Outlook 97 on Windows XP, in China, then we’d really like to hear from you. You don’t hear that much on Github.

NB The app deployed on Heroku currently sends the bouncebacks using the Mailtrap app, so you won’t actually receive the email, but I can give limited access to contributors so that they can see them all. I’ll truncate the Mailtrap inbox at irregular intervals.

What?

A Django app to make it easy to receive inbound emails from users via a hosted transactional email service (e.g. SendGrid, Postmark, Mandrill, etc.)

Contained within this project is the django app itself, together with a working Django project that uses the app, and is separately deployable to Heroku for testing purposes. The app has good test coverage, but it’s really very hard to test inbound emails without having real data, and that requires a public endpoint that you can use to hook up your preferred email provider’s webhooks.

Why?

If your project accepts inbound emails, you are probably using one of the big transactional email providers.

These services all provide a mechanism for receiving inbound emails which involves them (the service) parsing the inbound email and then posting the contents to an HTTP endpoint in your project. This is a great service, but it can often be fiddly to integrate into your app and it reinforces service lock-in, as each service’s callback is slightly different.

There is also a significant SMTP-HTTP ‘impedance mismatch’. You send emails through Django’s (SMTP) mail library, which provides the EmailMessage and EmailMultiAlternative objects, but you receive emails as an HTTP POST. Wouldn’t it be nice if you could both send and receive Django objects?

This app converts the incoming HttpRequest back into an EmailMultiAlternatives object, and fires a signal that sends both the new object, and the original request object. You simply have to listen for this signal, and process the email as your application requires.

The mail parser handles non-UTF-8 character sets (so those pesky PC Outlook emails don’t come through all garbled), and file attachments.

How?

Although this is Django app, it contains (for now) no models. Its principle component is a single view function that does the parsing. There is a single configuration setting - INBOUND_EMAIL_PARSER, which must be one of the supported backends.

This setting is expected to be available to the app from django.conf.settings, and the app will raise an error if it does not exist.

The default URL for inbound emails is simply ‘/inbound/’.

The flow through the app is very simple:

1. The app view function receive_inbound_email is called when a new email POST is received from your service provider.

2. This function looks up the INBOUND_EMAIL_PARSER, loads the appropriate backend, and parses the request.POST contents out into a new django.core.mail.EmailMultiAlternatives object.

3. The email_received signal is fired, and the new EmailMultiAlternatives instance is passed, along with the original HttpRequest (in case there’s any special handling that you require - e.g. DKIM / SPF info, if your provider passes that along).

If an email is unacceptable in some way (e.g. an attachment is too large), then the email_received_unacceptable signal is fired instead. This signal has an argument exception describing the problem.

Installation

For use as the app in Django project, use pip:

$ pip install django-inbound-email

For hacking on the project, pull from Git:

$ git pull [e-mail]:yunojuno/django-inbound-email.git
$ cd django-inbound-email
django-inbound-email$
# use virtualenvwrapper, and install Django to allow tests to run
django-inbound-email$ mkvirtualenv django-inbound-email
(django-inbound-email) django-inbound-email$ pip install django

Usage

Your main concern, after installing and configuring the app, is handling the email_received signal:

# This snippet goes somewhere inside your project,
# wherever you need to react to incoming emails.
import logging
from inbound_email.signals import email_received

def on_email_received(sender, **kwargs):
    """Handle inbound emails."""
    email = kwargs.pop('email')
    request = kwargs.pop('request')

    # your code goes here - save the email, respond to it, etc.
    logging.debug(
        "New email received from %s: %s",
        email.from_email,
        email.subject
    )

# pass dispatch_uid to prevent duplicates:
# https://docs.djangoproject.com/en/dev/topics/signals/
email_received.connect(on_email_received, dispatch_uid="something_unique")

Handling file attachments as FileField properties

There is one gotcha in the handling of file attachments. The email object that is sent via the signal has an attachments property, but this contains a list of 3-tuples [(name, contents, content_type),], not a list of file objects. In order to store the attachments against a model as a FileField, you’ll need to convert the tuples back into something that Django can deal with.

from django.core.files.uploadedfile import SimpleUploadedFile
from django.db import models

from inbound_email.signals import email_received


def get_file(attachment):
    """Convert email.attachment tuple into a SimpleUploadedFile."""
    name, content, content_type = attachment
    return SimpleUploadedFile(name, content, content_type)


class Example(models.Model):
    """Example model that contains a FileField property."""
    file = models.FileField()


def on_email_received(sender, **kwargs):
    """Handle inbound emails."""
    email = kwargs.pop('email')
    for attachment in email.attachments:
        # we must convert attachment tuple into a file
        # before adding as the property.
        example = Example(file=get_file(attachment))
        example.save()

Tests

There is a test django project, test_app that is used to run the tests.

(django-inbound-email) django-inbound-email$ python manage.py test

Configuration

• Install the app

• Add the app to INSTALLED_APPS

• Add INBOUND_EMAIL_PARSER setting

• Update your provider configuration to point to app URL

# the fully-qualified path to the provider's backend parser
INBOUND_EMAIL_PARSER = 'inbound_email.backends.sendgrid.SendGridRequestParser'

# if True (default=False) then log the contents of each inbound request
INBOUND_EMAIL_LOG_REQUESTS = True

# if True (default=True) then always return HTTP status of 200 (may be required by provider)
INBOUND_EMAIL_RESPONSE_200 = True

# add the app to Django's INSTALL_APPS setting
INSTALLED_APPS = (
    # other apps
    # ...
    'inbound_email',
)

Mandrill Features

If you wish to check the X-Mandrill-Signature header that Mandrill provides in the requests, you will need to set the INBOUND_MANDRILL_AUTHENTICATION_KEY setting to your Mandrill authentication key. When the key is supplied Inbound Email will check the signature supplied versus the one calculated from the request.

If signatures don’t match, the system will send the signal email_received_unacceptable with the exception describing the problem.

Features

Things it will do:

• Parse HTTP requests into EmailMultiAlternatives objects

• Pluggable backends (SendGrid, Mailgun and Mandrill currently supported)

• Handle character encodings properly

• Handle attachments, including if they are too large

Things it (probably) won’t do:

• Handle email reply parsing - use https://github.com/zapier/email-reply-parser