iot-edge-logger 1.0.2

IoT Edge Logger Library for Python

iot-edge-logger

This package is a custom log formatter to standardize, collect and analyze logs from IoT Edge Devices in an Azure Log Analytics Workspace.

Source code | Package PyPI

Table of Contents

• Getting Started
  • Prerequisites
  • Basic Examples
• Logging Considerations
  • Ideal Scenario
  • Example Architecture
• API Documentation
  • Init Logging Method
• Contributing
  • Local Development
  • Test Commands
• Versioning
• Deployment Process

Getting Started

This section provides basic examples with the iot-edge-logger.

Prerequisites

• Python 3.7 or later is required to use this package.

Basic Examples

1. Install via pip:

   pip install iot-edge-logger
   

2. Import and say hello:

   from iot.edge.logger import __version__
   
   
   print(f"hello world from iot-edge-logger version: {__version__}")
   

3. Basic usage:

   from iot.edge.logger import init_logging
   
   # setup logging
   logger = init_logging(module_name="my_iot_module")
   
   
   logger.info("I am alive!")
   logger.warning("Plotting global takeover...")
   logger.error("Humans have become suspicious, shutting down")
   

4. Log output:

   <6> 2022-08-02 19:04:13,015 [INF] my_iot_module I am alive!
   <4> 2022-08-02 19:04:13,015 [WRN] my_iot_module Plotting global takeover...
   <3> 2022-08-02 19:04:13,015 [ERR] my_iot_module Humans have become suspicious, shutting down
   

Logging Considerations

• Retrieve log methods from the edge agent are tightly coupled with syslog severity level standards. For the best compatibility with the log retrieval feature provided by Azure, the recommended logging format is implemented by default with this package.

• Some applications provide verbose logging, generating a lot of trace and information logs. While it's beneficial to know that your application is healthy, it is more important to take action when things go wrong and keep costs under control. Consider configuring with logs that match warning level or higher.

Ideal Scenario

The ideal scenario for leveraging iot-edge-logger would involve a custom module running on an IoT edge device.

Say you wrote a Python-based module for your edge device, but you're looking for a more centralized approach to logging - the default Azure UI to stream logs just isn't enough to scale your monitoring requirements.

By leveraging the same built-in retrieve logs method that the edge agent is using to display logs in Azure, we can define our own process for pulling, storing and analyzing edge device logs.

Example Architecture

An example architecture of the ideal scenario defined above may look something like this:

And querying in log analytics would look something like this:

iotedgemodulelogs
| where timestamp_t <= ago(24h)
| project timestamp_t, moduleId_s, logLevel_d, Message

API Documentation

Init Logging Method

A custom logger to provide well formatted logging for a given IoT Edge Module - or any Python program.

init_logging(
    module_name,
    level="DEBUG",
    format="<%(levelno)s> %(asctime)s [%(levelname)s] %(module_name)s %(message)s",
    datefmt=None,
    timespec="milliseconds",
    timezone="UTC",
)

Parameters

• module_name str

  The name of the IoT Edge module that is using this logger. Used for filtering and querying in log analytics - use "this_format_for_module".

• level Optional[str]

  The logging level. Default is "DEBUG".

• format Optional[str]

  The desired logging format. Default seen above.

• datefmt Optional[str]

  The desired date format. Default is None.

• timespec Optional[str]

  The time accuracy represented in log records. Default is "milliseconds".

• timezone Optional[str]

  The timezone represented in log records. Default is "UTC".

Valid Timezones

• US/Alaska

• US/Central

• US/Eastern

• US/Mountain

• US/Pacific

• UTC

Returns

Returns a logger object.

Contributing

Contributions and suggestions are welcomed. However, there is a level of responsibility placed on the contributor to follow best-practices, provide thorough testing, follow the branching strategy, use the pull request template, and maintain a positive and coachable attitude when receiving feedback or questions on your code.

Local Development

1. Clone the repository.

2. Install and enable pre-commit

3. Checkout to a features/* or issue/* branch.

4. Create and activate venv.

5. Develop.

6. Write, test and validate unit tests.

7. Submit PR and comply with PR template.

8. Feedback loop and review process.

9. Thanks for your help!

Test Commands

To run your tests with the default unittest library:

python3 -m unittest discover tests

Versioning

This repository adheres to Semantic Versioning. It will be maintained through the CHANGELOG.md - as is standard with PyPI packages It's important to note that you must maintain the version with your releases in iot/edge/logger/_version.py, otherwise a new package version will fail to get published.

Deployment Process

1. Linting, testing and building occurs when a pull request is made from a features/* branch to the master branch.

2. Deployments to PyPI occur when an approved user triggers the GitHub Action. If the version has not been updated, this deployment will fail.