herodotus-logger 1.0.0

An enhanced python logger

Herodotus

An awesome enhanced python logger
Explore the docs »

Examples · Report Bug · Request Feature

Table of Contents

1. About The Project
   • The history behind the project
   • The naming convention
2. Getting Started
   • Installation
3. Usage
   • Basic usage (no formatter)
   • Use with a Formatter
   • Using the colorizer
4. Contributing
5. License
6. Contact

About The Project

The history behind the project

Python logging package is a powerful tool for logging your messages into any stream. From a file to third party service, like Elasticsearch.

But one day in a project I needed to log some messages through many streams with different severities. For example I want a logger object in which three handlers one for a rotating file handler, one for stdout and one for elasticsearch. But I wanted to send each severity to a specific stream. And also I wanted to colorize the stdout log but not the one in the file.

When I used something like this:

import logging
import some_colorizer_function

my_logger = logging.getLogger("my_logger")
my_logger.debug(
    some_colorizer_function("some message with %s"),
    "args"
)

It sends a nice colorized output to the stdout. But an ugly text (with ansi color symbols) sent to the Elasticsearch or written in the file. So I decided to make some enhancements on the logging package and also decided to publish it. So please fill free to contribute.

The naming convention

Herodotus was an ancient Greek historian known as the "Father of History." He wrote the book "The Histories," one of the earliest historical works. This book covers various subjects such as history, geography, cultures, civilizations, and wars. He combined accurate descriptions of events with engaging storytelling. His work is a blend of historical analysis and cultural narratives, making him a significant figure in the development of historical writing.

Getting Started

I've also created a pypi package for this library. So you can easily use and install it with pip or clone the project.

Installation

pip install herodotus_logger --upgrade

Usage

Basic usage

1. First, you should define a logger object with a specific severity level. by this level setup, the logger will send all severities greater than or equal to.You can read more about severity numbers here. For example, if you define a logger object with WARNING level, it does not send any INFO, DEBUG or NOTSET levels to its handlers.

   import logging
   from herodotus import logger
    
   lg = logger.Logger(
        name="test_logger",
        level=logging.WARNING
   )
   

2. You also should give it some handlers. You have two main options to do so:

   1. Use some basic provided handlers in the herodotus.handlers which are starting with Enhanced*
      • Note that all provided handlers' arguments are as the main one. They just accept some more arguments I'll explain.
   2. Use any custom or other handlers which are of type Handler in python.

   import logging
   from sys import stdout
   
   from herodotus import logger
   from herodotus import handlers
   
   lg = logger.Logger(
       name="test_logger",
       level=logging.WARNING,
       handlers=[
           handlers.EnhancedStreamHandler(
               stream=stdout,
               level=logging.WARNING
           ),
           handlers.EnhancedFileHandler(
               filename="logs/test_logfile.log",
               mode="a",
               encoding="utf-8",
               level=logging.CRITICAL
           )
       ]
   )
   

3. It's finished!! Seat back and just tell your logger object to log!

   1. Create the logs directory:

      mkdir logs
      

   2. Call the logger logs functions (ex debug, info,...)

      lg.logger.info("Hello")
      

   But at this point, nothing happened. Because the lg log level is logging.WARNING and we tell to log with info level. And we know that log.INFO < log.WARNING.

   Let's try another one:

   lg.logger.warning("Hello")
   

   and the bash output is:

   2023-08-09T10:39:05|test_logger|WARNING|Hello
   

   But nothing logged in the log file. And the reason is clear.

   Let's run another example:

   lg.logger.critical("Hello")
   

   and the bash output is:

   2023-08-09T10:45:45|test_logger|CRITICAL|Hello
   

   And finally in the log file located at logs/test_logfile.log we have the same output.

Use with a Formatter

I define a default formatter for the logger as follow:

self.formatter = formatter or logging.Formatter(
    datefmt="%Y-%m-%dT%H:%M:%S",
    fmt="%(asctime)s|%(name)s|%(levelname)s|%(message)s"
)

But you can change it when you create the logger:

import logging
from sys import stdout

from herodotus import logger
from herodotus import handlers

lg = logger.Logger(
    name="test_logger",
    level=logging.WARNING,
    formatter=logging.Formatter(
        datefmt="%Y-%m-%dT%H:%M:%S",
        fmt="%(asctime)s %(levelname)s: %(message)s"
    ),
    handlers=[
        handlers.EnhancedStreamHandler(
            stream=stdout,
            level=logging.WARNING
        ),
        handlers.EnhancedFileHandler(
            filename="logs/test_logfile.log",
            mode="a",
            encoding="utf-8",
            level=logging.CRITICAL
        )
    ]
)

The most important note is that you can also set different formatter for each handler. But if you don't set a formatter for your handler, the logger will use its formatter for.

import logging
from sys import stdout

from herodotus import logger
from herodotus import handlers

lg = logger.Logger(
    name="test_logger",
    level=logging.WARNING,
    formatter=logging.Formatter(
        datefmt="%Y-%m-%dT%H:%M:%S",
        fmt="%(asctime)s %(levelname)s: %(message)s"
    ),
    handlers=[
        handlers.EnhancedStreamHandler(
            stream=stdout,
            level=logging.WARNING
        ),
        handlers.EnhancedFileHandler(
            filename="logs/test_logfile.log",
            mode="a",
            encoding="utf-8",
            level=logging.CRITICAL,
            formatter=logging.Formatter(
                datefmt="%H:%M:%S",
                fmt="%(asctime)s: %(message)s"
            )
        )
    ]
)

Using the colorizer

Using colors everywhere undoubtedly gives another view, in the logging so. You can use colored package. But I also put some easy to use functions to add colors to your logs. Let's see some examples:

import logging
from sys import stdout

from herodotus import logger
from herodotus import handlers
from herodotus.utils import colorizer

lg = logger.Logger(
    name="test_logger",
    level=logging.WARNING,
    formatter=logging.Formatter(
        datefmt="%Y-%m-%dT%H:%M:%S",
        fmt="%(asctime)s %(levelname)s: %(message)s"
    ),
    handlers=[
        handlers.EnhancedStreamHandler(
            stream=stdout,
            level=logging.WARNING
        )
    ]
)

lg.logger.critical(colorizer.colorize("Hello", foreground="green"))

and the output will be something like this:

You can also add styles (as noted in the colored package). To do so, just pass your desired styles as a list to the colorize function:

lg.logger.critical(colorizer.colorize("Hello", foreground="green", styles=['bold', 'underline']))

And the output will be something like this:

But what happens if we add a file handler to a logger which uses the colorize function? Let's see:

import logging
from sys import stdout

from herodotus import logger
from herodotus import handlers
from herodotus.utils import colorizer

lg = logger.Logger(
    name="test_logger",
    level=logging.WARNING,
    formatter=logging.Formatter(
        datefmt="%Y-%m-%dT%H:%M:%S",
        fmt="%(asctime)s %(levelname)s: %(message)s"
    ),
    handlers=[
        handlers.EnhancedStreamHandler(
            stream=stdout,
            level=logging.WARNING
        ),
        handlers.EnhancedFileHandler(
            filename="logs/test_logfile.log",
            mode="a",
            encoding="utf-8",
            level=logging.CRITICAL,
            formatter=logging.Formatter(
                datefmt="%H:%M:%S",
                fmt="%(asctime)s: %(message)s"
            )
        )
    ]
)

lg.logger.critical(colorizer.colorize("Hello", foreground="green"))

In the log file, you will probably see something like this (If you don't have any plugin or extention to convert ansii chars to the colors):

So ugly! So what to do? Don't be worry. I also have a soloution for this.

You can use the msg_func argument for each of Enhanced* handlers. It has the type of function So, you should pass it a function. I, for example, have written a decolorize function in the herodotus.utils.colorize package which get a str with ansii color chars and remove them:

handlers.EnhancedFileHandler(
    filename="logs/test_logfile.log",
    mode="a",
    encoding="utf-8",
    level=logging.CRITICAL,
    msg_func=colorizer.decolorize,
    formatter=logging.Formatter(
        datefmt="%H:%M:%S",
        fmt="%(asctime)s: %(message)s"
    )

lg.logger.critical(colorizer.colorize("Hello", foreground="green"))

Finally, in the log file you will see something like this:

See the open issues for a full list of proposed features( and known issues).

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement." Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the MIT License. See LICENSE.txt for more information.

Contact

Emad Helmi | Find me on Twitter @EmadHelmi

Or send me Email [e-mail]

MIT License

Copyright (c) [2023] [Herodotus]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.