Skip to main content

A library which add DBHandler to Django logging handlers.

Project description

django-camel-spitter

codecov

Hi. I am a very rude camel 🐫 and I like to spit logs 💦 directly into your database 🗄️.

Introduction

Project django-camel-spitter adds a new handler to standard django logging system.

Purpose of this handler is to store logs straight to the database, primary with simplicity and opportunity to easy extend this solution.

Installation

# pip
pip install django-camel-spitter

# pipenv
pipenv install django-camel-spitter

# poetry
poetry add django-camel-spitter

Setup

1. Adding camel_spitter to settings.INSTALLED_APPS:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'camel_spitter'
)

2. Adding additional logging db connection to settings.DATABASES:

This additional connection is needed for handling DB transaction atomicity. Exception without own connection cannot create DB log, when rollback was made. Transactions in django are default handled on default DB connection. So rollback will stop default connection execution, but logging connection not.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'HOST': os.getenv('DATABASE_HOST'),
        'PORT': os.getenv('DATABASE_PORT', 5432),
        'NAME': os.getenv('DATABASE_NAME'),
        'USER': os.getenv('DATABASE_USER'),
        'PASSWORD': os.getenv('DATABASE_PASSWORD', None)
    },
    'logging': {
        'ENGINE': 'django.db.backends.postgresql',
        'HOST': os.getenv('DATABASE_HOST'),
        'PORT': os.getenv('DATABASE_PORT', 5432),
        'NAME': os.getenv('DATABASE_NAME'),
        'USER': os.getenv('DATABASE_USER'),
        'PASSWORD': os.getenv('DATABASE_PASSWORD', None)
    }   
}

3. Adding model, filter and handler to settings.LOGGING:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'filters': {
        'db_filter': {
            '()': 'camel_spitter.db_filter.DBFilter',
        },
    },
    'handlers': {
        'db': {
            'level': 'INFO',
            'class': 'camel_spitter.db_handler.DBHandler',
            'model': 'tests.models.BasicLogEntry',  # path to your custom model
            'filters': ['db_filter']
        }
    },
    'loggers': {
        'logger': {
            'handlers': ['db'],
            'level': 'INFO'
        }
    }
}

4. Creating a log model:

Only importance is inheritance from camel_spitter.models.BaseLogModel.

from camel_spitter.models import BaseLogModel


class BasicLogEntry(BaseLogModel):
    class Meta:
        app_label = 'tests'
        db_table = 'log_entries'
        default_permissions = ()

Example

1. Quick use

If you did all setup steps, you are ready to log to the database.

import logging
from app.models import BasicLogEntry

logging.getLogger('logger').error('Foo Bar Error')
logged_information = BasicLogEntry.objects.get(message='Foo Bar Error')

# logged_information = {BasicLogEntry}BasicLogEntry object (1)

2. Example of extended model

If you like to log some additional data, for example: [request.body, user_name], you need to first add these fields to model:

from camel_spitter.models import BaseLogModel
from django.db import models
from django.contrib.postgres.fields import JSONField

class ExtendedLogEntry(BaseLogModel):
    class Meta:
        app_label = 'tests'
        db_table = 'extended_log_entries'
        default_permissions = ()

    request_body = JSONField(null=True)
    user_name = models.CharField(max_length=100, null=True)

As a second, you need to add these additional data to logging:

import logging

logging.getLogger('logger').error('Foo Bar Error', extra={
    'request_body': json.loads(request.body), 'user_name': 'Foo Bar'
})

Important notes

Testing with pytest and file DB

When tests are made with pytest library and file DB like SQLite, tests will make two separate database files from specified connections even though path and engine are the same. So retrieving log is needed to be executed with using like BasicLogEntry.objects.using('logging').get(message='Foo Bar'). To avoid this issue you can specify mirror database in test settings.

    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': DB_PATH,
    },
    'logging': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': DB_PATH,
        'TEST': {
            'MIRROR': 'default',
        }
    },

Made with ❤ by Adam Žúrek & BACKBONE s.r.o.

0.1.0 : 2020-06-27

  • Initial release

0.2.0 : 2020-06-29

  • Changed python version requirement from "^0.8" to "^0.6"

0.3.0 : 2020-08-04

  • DB logging is successful even if it is wrapped in a transaction, which made rollback.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-camel-spitter-0.3.0.tar.gz (5.0 kB view hashes)

Uploaded Source

Built Distribution

django_camel_spitter-0.3.0-py3-none-any.whl (6.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page