structlog-sentry-logger 0.7.1

Log without the setup via a pre-configured structlog logger with optional Sentry integration

Structlog-Sentry-Logger

Overview

A multi-purpose, pre-configured, performance-optimized structlog logger with (optional) Sentry integration via structlog-sentry.

Features

1. Makes logging as easy as using print statements, but prettier and less smelly!
2. Highly opinionated! There are only two (2) distinct configurations.
3. Structured logs in JSON format means they are ready to be ingested by many of your favorite log analysis tools!

What You Get

Powerful Automatic Context Fields

The pre-configured options include:

1. Timestamps
   • DATETIME_FORMAT = "%Y-%m-%d %H:%M:%S"
2. Log levels
   • Added to the JSON context for filtering and categorization
3. Logger names
   • Automatically assigned to namespaced versions of the initializing python modules (.py files), relative to your project directory.
     • e.g., the logger in docs_src/sentry_integration.py is named docs_src.sentry_integration

With fields sorted by key for easier at-a-glance analysis.

Performance

structlog-sentry-logger is fully-tuned and leverages ORJSON as the JSON serializer for lightning-fast logging (more than a 4x speedup over Python's built-in JSON library[1]). It's 2021, you don't have to let your obligate cross-cutting concerns cripple performance any longer!

For further reference, see:

• "ORJSON: Serialize" for benchmarks
• "structlog: Performance" for salient performance-related configurations.

[1] source: Choosing a faster JSON library for Python: Benchmarking

Built-in Sentry Integration (Optional)

Automatically add much richer context to your Sentry reports.

• Your entire logging context is sent as a Sentry event when the structlog-sentry-logger log level is error or higher.
  • i.e., logger.error(""), logger.exception("")
• See structlog-sentry for more details.

---

Table of Contents

• Installation
• Usage
  • Pure structlog Logging (Without Sentry)
  • Sentry Integration
    • Log Custom Context Directly to Sentry
• Output: Formatting & Storage
• Summary
• Further Reading
  • structlog
  • Sentry
  • structlog-sentry

Installation

pip install structlog-sentry-logger

Usage

Pure structlog Logging (Without Sentry)

At the top of your Python module, import and instantiate the logger:

import structlog_sentry_logger

LOGGER = structlog_sentry_logger.get_logger()

Now anytime you want to print anything, don't. Instead do this:

LOG_MSG = "Information that's useful for future me and others"
LOGGER.info(LOG_MSG, extra_field="extra_value")

:note: Note
All the regular Python logging levels are supported.

Which automatically produces this:

{
    "event": "Information that's useful for future me and others",
    "extra_field": "extra_value",
    "level": "info",
    "logger": "docs_src.pure_structlog_logging_without_sentry",
    "sentry": "skipped",
    "timestamp": "2020-10-18 15:30:05"
}

Sentry Integration

Export your Sentry DSN into your local environment.

• An easy way to do this is to put it into a local .env file and use python-dotenv to populate your environment:

# On the command line:
SENTRY_DSN=YOUR_SENTRY_DSN
echo "SENTRY_DSN=${SENTRY_DSN}" >> .env

Then load the .env file in your Python code prior to instantiating the logger, e.g.:

from dotenv import find_dotenv, load_dotenv
load_dotenv(find_dotenv())

import structlog_sentry_logger
LOGGER = structlog_sentry_logger.get_logger()

Log Custom Context Directly to Sentry

With structlog, you can even incorporate custom messages in your exception handling which will automatically be reported to Sentry (thanks to the structlog-sentry module):

import uuid

import structlog_sentry_logger

LOGGER = structlog_sentry_logger.get_logger()

curr_user_logger = LOGGER.bind(uuid=uuid.uuid4().hex)  # LOGGER instance with bound UUID
try:
    curr_user_logger.warn("A dummy error for testing purposes is about to be thrown!")
    x = 1 / 0
except ZeroDivisionError as err:
    ERR_MSG = (
        "I threw an error on purpose for this example!\n"
        "Now throwing another that explicitly chains from that one!"
    )
    curr_user_logger.exception(ERR_MSG)
    raise RuntimeError(ERR_MSG) from err

{
    "event": "A dummy error for testing purposes is about to be thrown!",
    "level": "warning",
    "logger": "docs_src.sentry_integration",
    "sentry": "skipped",
    "timestamp": "2020-10-18 15:29:55",
    "uuid": "181e0e00b9034732af4fed2b8424fb11"
}
{
    "event": "I threw an error on purpose for this example!\nNow throwing another that explicitly chains from that one!",
    "exception": 'Traceback (most recent call last):\n  File "/app/structlog-sentry-logger/docs_src/sentry_integration.py", line 10, in <module>\n    x = 1 / 0\nZeroDivisionError: division by zero',
    "level": "error",
    "logger": "docs_src.sentry_integration",
    "sentry": "sent",
    "sentry_id": null,
    "timestamp": "2020-10-18 15:29:55",
    "uuid": "181e0e00b9034732af4fed2b8424fb11"
}
Traceback (most recent call last):
  File "/app/structlog-sentry-logger/docs_src/sentry_integration.py", line 10, in <module>
    x = 1 / 0
ZeroDivisionError: division by zero

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/structlog-sentry-logger/docs_src/sentry_integration.py", line 17, in <module>
    raise RuntimeError(ERR_MSG) from err
RuntimeError: I threw an error on purpose for this example!
Now throwing another that explicitly chains from that one!

Output: Formatting & Storage

The default behavior is to stream JSON logs directly to the standard output stream like a proper 12 Factor App.

For local development, it often helps to prettify logging to stdout and save JSON logs to a .logs folder at the root of your project directory for later debugging. To enable this behavior, set the following environment variable (e.g. via python-dotenv as in the Sentry Integration section):

CI_ENVIRONMENT_SLUG=dev-local

In doing so, with our previous exception handling example we would get:

Development

📝 Note
For convenience, many of the below processes are abstracted away and encapsulated in single Make targets.

🔥 Tip
Invoking make without any arguments will display auto-generated documentation on available commands.

Package and Dependencies Installation

Make sure you have Python 3.6+ and poetry installed and configured.

To install the package and all dev dependencies, run:

make provision_environment

🔥 Tip
Invoking the above without poetry installed will emit a helpful error message letting you know how you can install poetry.

Testing

We use tox for our test automation framework and pytest for our testing framework.

To invoke the tests, run:

make test

Run mutation tests to validate test suite robustness (Optional):

make test-mutations

📝 Note
Test time scales with the complexity of the codebase. Results are cached in .mutmut-cache, so once you get past the initial cold start problem, subsequent mutation test runs will be much faster; new mutations will only be applied to modified code paths.

Code Quality

We are using pre-commit for our code quality static analysis automation and management framework.

To invoke the analyses and auto-formatting over all version-controlled files, run:

make lint

🚨 Danger
CI will fail if either testing or code quality fail, so it is recommended to automatically run the above locally prior to every commit that is pushed.

Automate via Git Pre-Commit Hooks

To automatically run code quality validation on every commit (over to-be-committed files), run:

make install-pre-commit-hooks

⚠️ Warning
This will prevent commits if any single pre-commit hook fails (unless it is allowed to fail) or a file is modified by an auto-formatting job; in the latter case, you may simply repeat the commit and it should pass.

Documentation

make docs-clean docs-html

📝 Note
For faster feedback loops, this will attempt to automatically open the newly built documentation static HTML in your browser.

Summary

That's it. Now no excuses. Get out there and program with pride knowing no one will laugh at you in production! For not logging properly, that is. You're on your own for that other observability stuff.

Further Reading

structlog: Structured Logging for Python

Sentry: Monitor and fix crashes in realtime.

structlog-sentry: Provides the structlog SentryProcessor for Sentry integration.

---

Legal

License

Structlog-Sentry-Logger is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Credits

This project was generated from @TeoZosa's cookiecutter-cruft-poetry-tox-pre-commit-ci-cd template.