moesifdjango 1.7.4

Moesif Middleware for Python Django

Moesif Middleware for Python Django

Django middleware to log incoming API calls hitting your own service or outgoing API calls going out to third parties and sends to Moesif for API analytics and log analysis.

Source Code on GitHub

This SDK uses the Requests library and will work for Python 2.7 — 3.5.

For high volume APIs, you can enable Celery which offloads the logging in a separate task. Install celery via pip install celery and set USE_CELERY to True

How to install

pip install moesifdjango

How to use

In your settings.py file in your Django project directory, please add moesifdjango.middleware.moesif_middleware to the MIDDLEWARE array. If you plan to use celery as the backend of asynchronous delivered logged requests, you also need to add moesifdjango to your INSTALLED_APPS.

Because of middleware execution order, it is best to add moesifdjango middleware below SessionMiddleware and AuthenticationMiddleware, because they add useful session data that enables deeper error analysis. On the other hand, if you have other middleware that modified response before going out, you may choose to place Moesif middleware above the middleware modifying response. This allows Moesif to see the modifications to the response data and see closer to what is going over the wire.

Changes in Django 1.10

Django middleware style and setup was refactored in version 1.10. You need need to import the correct version of Moesif middleware depending on your Django version. If you're using Django 1.10 or greater, use moesifdjango.middleware.moesif_middleware. However, if you're using Django 1.9 or older, you need to follow the legacy style for importing middleware and use moesifdjango.middleware_pre19.MoesifMiddlewarePre19 instead.

You can find your current Django version via python -c "import django; print(django.get_version())" {: .notice--info}

Django 1.10 or newer

Add the middleware to your application:

Django 1.10 renamed MIDDLEWARE_CLASSES to MIDDLEWARE. If you're using 1.10 or newer and still using the legacy MIDDLEWARE_CLASSES, the Moesif middleware will not run. {: .notice--danger}

MIDDLEWARE = [
    ...
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'moesifdjango.middleware.moesif_middleware'
    ...
]

Django 1.9 or older

Add the middleware to your application:

MIDDLEWARE_CLASSES = [
    ...
    'moesifdjango.middleware_pre19.MoesifMiddlewarePre19',
    ...
    # other middlewares
]

Also, add MOESIF_MIDDLEWARE to your settings.py file,

MOESIF_MIDDLEWARE = {
    'APPLICATION_ID': 'Your Moesif Application Id',
    'LOG_BODY': True,
    ...
    # For other options see below.
}

Your Moesif Application Id can be found in the Moesif Portal. After signing up for a Moesif account, your Moesif Application Id will be displayed during the onboarding steps.

You can always find your Moesif Application Id at any time by logging into the Moesif Portal, click on the top right menu, and then clicking Installation.

Configuration options

APPLICATION_ID

(required), string, is obtained via your Moesif Account, this is required.

Options specific to incoming API calls

SKIP

(optional) (request, response) => boolean, a function that takes a request and a response, and returns true if you want to skip this particular event.

IDENTIFY_USER

(optional) (request, response) => string, a function that takes a request and a response, and returns a string that is the user id used by your system. While Moesif identify users automatically, and this middleware try to use the standard Django request.user.username, if your set up is very different from the standard implementations, it would be helpful to provide this function.

IDENTIFY_COMPANY

(optional) (request, response) => string, a function that takes a request and a response, and returns a string that is the company id for this event.

GET_SESSION_TOKEN

(optional) (request, response) => string, a function that takes a request and a response, and returns a string that is the session token for this event. Again, Moesif tries to get the session token automatically, but if you setup is very different from standard, this function will be very help for tying events together, and help you replay the events.

GET_METADATA

(optional) (request, response) => dictionary, getMetadata is a function that returns an object that allows you to add custom metadata that will be associated with the event. The metadata must be a dictionary that can be converted to JSON. For example, you may want to save a VM instance_id, a trace_id, or a tenant_id with the request.

LOG_BODY

(optional) boolean, default True, Set to False to remove the HTTP body before sending to Moesif. If you want more control over which fields are included or not included look at the individual mask method below.

MASK_EVENT_MODEL

(optional) (EventModel) => EventModel, a function that takes an EventModel and returns an EventModel with desired data removed. Use this if you prefer to write your own mask function than use the string based filter options: REQUEST_BODY_MASKS, REQUEST_HEADER_MASKS, RESPONSE_BODY_MASKS, & RESPONSE_HEADER_MASKS. The return value must be a valid EventModel required by Moesif data ingestion API. For details regarding EventModel please see the Moesif Python API Documentation.

Options specific to outgoing API calls

CAPTURE_OUTGOING_REQUESTS

boolean, Default False. Set to True to capture all outgoing API calls from your app to third parties like Stripe or to your own dependencies while using Requests library. The options below is applied to outgoing API calls. When the request is outgoing, for options functions that take request and response as input arguments, the request and response objects passed in are Requests request or response objects.

SKIP_OUTGOING

(optional) (req, res) => boolean, a function that takes a Requests request and response, and returns true if you want to skip this particular event.

IDENTIFY_USER_OUTGOING

(optional, but highly recommended) (req, res) => string, a function that takes Requests request and response, and returns a string that is the user id used by your system. While Moesif tries to identify users automatically, but different frameworks and your implementation might be very different, it would be helpful and much more accurate to provide this function.

IDENTIFY_COMPANY_OUTGOING

(optional) (req, res) => string, a function that takes Requests request and response, and returns a string that is the company id for this event.

GET_METADATA_OUTGOING

(optional) (req, res) => dictionary, a function that takes Requests request and response, and returns a dictionary (must be able to be encoded into JSON). This allows to associate this event with custom metadata. For example, you may want to save a VM instance_id, a trace_id, or a tenant_id with the request.

GET_SESSION_TOKEN_OUTGOING

(optional) (req, res) => string, a function that takes Requests request and response, and returns a string that is the session token for this event. Again, Moesif tries to get the session token automatically, but if you setup is very different from standard, this function will be very help for tying events together, and help you replay the events.

General options

LOG_BODY_OUTGOING

(optional) boolean, default True, Set to False to remove logging request and response body.

USE_CELERY

boolean, Default False. Set to True to use Celery for queuing sending data to Moesif. Check out Celery documentation for more info.

How to use Celery

Because celery is optional, moesifdjango does not prepackage Celery as a dependency. Make sure you install celery via pip install celery

Install celery and redis with pip install "celery[redis]"

Please Note: If you're using Celery 3.1 or earlier, install celery and redis with pip install celery==3.1.25 and pip install redis==2.10.6

Set the configuration option to USE_CELERY to True.

MOESIF_MIDDLEWARE = {
    'USE_CELERY': True
}

Start the celery worker with celery -A <projectName> worker --loglevel=debug

LOCAL_DEBUG

boolean, set to True to print internal log messages for debugging SDK integration issues.

Example

A more detailed example is available at https://github.com/Moesif/moesifdjangoexample

def identify_user(req, res):
    # Your custom code that returns a user id string
    if req.user and req.user.is_authenticated:
        return req.user.username
    else:
        return None

def identify_company(req, res):
    # Your custom code that returns a company id string
    return "67890"

def should_skip(req, res):
    # Your custom code that returns true to skip logging
    return "health/probe" in req.path

def get_token(req, res):
    # If you don't want to use the standard Django session token,
    # add your custom code that returns a string for session/API token
    return "XXXXXXXXXXXXXX"

def mask_event(eventmodel):
    # Your custom code to change or remove any sensitive fields
    if 'password' in eventmodel.response.body:
        eventmodel.response.body['password'] = None
    return eventmodel

def get_metadata(req, res):
    return {
        'datacenter': 'westus',
        'deployment_version': 'v1.2.3',
    }


MOESIF_MIDDLEWARE = {
    'APPLICATION_ID': 'Your application id',
    'LOCAL_DEBUG': False,
    'LOG_BODY': True,
    'IDENTIFY_USER': identify_user,
    'IDENTIFY_COMPANY': identify_company,
    'GET_SESSION_TOKEN': get_token,
    'SKIP': should_skip,
    'MASK_EVENT_MODEL': mask_event,
    'GET_METADATA': get_metadata,
    'USE_CELERY': False
}

Update User

Update A Single User

Create or update a user profile in Moesif. The metadata field can be any customer demographic or other info you want to store. Only the user_id field is required. This method is a convenient helper that calls the Moesif API lib. For details, visit the Python API Reference.

middleware = MoesifMiddleware(None)

# Only user_id is required.
# Campaign object is optional, but useful if you want to track ROI of acquisition channels
# See https://www.moesif.com/docs/api#users for campaign schema
# metadata can be any custom object
user = {
  'user_id': '12345',
  'company_id': '67890', # If set, associate user with a company object
  'campaign': {
    'utm_source': 'google',
    'utm_medium': 'cpc', 
    'utm_campaign': 'adwords',
    'utm_term': 'api+tooling',
    'utm_content': 'landing'
  },
  'metadata': {
    'email': '[e-mail]',
    'first_name': 'John',
    'last_name': 'Doe',
    'title': 'Software Engineer',
    'sales_info': {
        'stage': 'Customer',
        'lifetime_value': 24000,
        'account_owner': '[e-mail]'
    },
  }
}

update_user = middleware.update_user(user)

Update Users in Batch

Similar to update_user, but used to update a list of users in one batch. Only the user_id field is required. This method is a convenient helper that calls the Moesif API lib. For details, visit the Python API Reference.

middleware = MoesifMiddleware(None)

userA = {
  'user_id': '12345',
  'company_id': '67890', # If set, associate user with a company object
  'metadata': {
    'email': '[e-mail]',
    'first_name': 'John',
    'last_name': 'Doe',
    'title': 'Software Engineer',
    'sales_info': {
        'stage': 'Customer',
        'lifetime_value': 24000,
        'account_owner': '[e-mail]'
    },
  }
}

userB = {
  'user_id': '54321',
  'company_id': '67890', # If set, associate user with a company object
  'metadata': {
    'email': '[e-mail]',
    'first_name': 'Mary',
    'last_name': 'Jane',
    'title': 'Software Engineer',
    'sales_info': {
        'stage': 'Customer',
        'lifetime_value': 48000,
        'account_owner': '[e-mail]'
    },
  }
}
update_users = middleware.update_users_batch([userA, userB])

Update Company

Update A Single Company

Create or update a company profile in Moesif. The metadata field can be any company demographic or other info you want to store. Only the company_id field is required. This method is a convenient helper that calls the Moesif API lib. For details, visit the Python API Reference.

middleware = MoesifMiddleware(None)

# Only company_id is required.
# Campaign object is optional, but useful if you want to track ROI of acquisition channels
# See https://www.moesif.com/docs/api#update-a-company for campaign schema
# metadata can be any custom object
company = {
  'company_id': '67890',
  'company_domain': 'acmeinc.com', # If domain is set, Moesif will enrich your profiles with publicly available info 
  'campaign': {
    'utm_source': 'google',
    'utm_medium': 'cpc', 
    'utm_campaign': 'adwords',
    'utm_term': 'api+tooling',
    'utm_content': 'landing'
  },
  'metadata': {
    'org_name': 'Acme, Inc',
    'plan_name': 'Free',
    'deal_stage': 'Lead',
    'mrr': 24000,
    'demographics': {
        'alexa_ranking': 500000,
        'employee_count': 47
    },
  }
}

update_company = middleware.update_company(company)

Update Companies in Batch

Similar to update_company, but used to update a list of companies in one batch. Only the company_id field is required. This method is a convenient helper that calls the Moesif API lib. For details, visit the Python API Reference.

middleware = MoesifMiddleware(None)

companyA = {
  'company_id': '67890',
  'company_domain': 'acmeinc.com', # If domain is set, Moesif will enrich your profiles with publicly available info 
  'metadata': {
    'org_name': 'Acme, Inc',
    'plan_name': 'Free',
    'deal_stage': 'Lead',
    'mrr': 24000,
    'demographics': {
        'alexa_ranking': 500000,
        'employee_count': 47
    },
  }
}

companyB = {
  'company_id': '09876',
  'company_domain': 'contoso.com', # If domain is set, Moesif will enrich your profiles with publicly available info 
  'metadata': {
    'org_name': 'Contoso, Inc',
    'plan_name': 'Free',
    'deal_stage': 'Lead',
    'mrr': 48000,
    'demographics': {
        'alexa_ranking': 500000,
        'employee_count': 53
    },
  }
}

update_companies = middleware.update_companies_batch([userA, userB])

Tested versions

Moesif has validated moesifdjango against the following combinations. Using the Celery queing service is optional, but can be enabled to enable higher performance.

Python	Django	Celery	Redis	Test with Celery	Test w/o Celery
Python 2.7	1.11.22	3.1.25	2.10.6	Yes	Yes
Python 2.7	1.11.22	4.3.0	3.2.1	Yes	Yes
Python 2.7	1.9				Yes
Python 3.4.5	1.11.22	3.1.25	2.10.6		Yes
Python 3.4.5	1.11.22	4.3.0	3.2.1		Yes
Python 3.4.5	1.9			Yes
Python 3.6.4	1.11.22	3.1.25	2.10.6	Yes	Yes
Python 3.6.4	1.11.22	4.3.0	3.2.1	Yes	Yes
Python 3.6.4	1.9				Yes

How to test

1. Manually clone the git repo
2. Invoke pip install Django if you haven't done so.
3. Invoke pip install moesifdjango
4. Add your own application id to 'tests/settings.py'. You can find your Application Id from Moesif Dashboard -> Top Right Menu -> Installation
5. From terminal/cmd navigate to the root directory of the middleware tests.
6. Invoke python manage.py test if you are using Django 1.10 or newer.
7. Invoke python manage.py test middleware_pre19_tests if you are using Django 1.9 or older.

Example

An example Moesif integration based on quick start tutorials of Django and Django Rest Framework: Moesif Django Example

Other integrations

To view more documentation on integration options, please visit the Integration Options Documentation.