Skip to main content

Faster logger than python native logging package.

Project description

MH370

better logger for python

Performance of existing packages

Setup:
Computer - Apple Macbook air M1
Operating system - iOS Sonama 14.6.1
Python version - 3.12.6

1000 iterations

Package time taken
without any logger 0.001s
logging 2.11s
loguru 15.19s
structlog 9.81s
lo3 v0.2 2.84s
lo3 v0.3 0.41s

10,000 iterations

Package time taken
without any logger 0.01s
logging 21.14s
loguru 154.53s
structlog 62.62s
lo3 v0.2 28.59s
lo3 v0.3 3.98s

250,000 iterations

Package time taken
logging 522.20
lo3 v3 101.39

Without a doubt every logger has it's own set of overhead in the code performance. Let's see whether I can improve this. RustLogger gave a whopping ~80% performance improvement in both high and low volume testing.


Lo3 v0.3.0

Optimized rust code a bit to reduce redundant unwanted operations,

  • Instead of opening file for every entry, now it is being done for once, at instantiation of the Logger
  • Replaced += with push_str to append the logs
  • BufWriter to reduce number of system calls
  • built the package with maturin develop --release

Primliminary scope (No change)

  • Adds log to file
  • Provides info, warn, debug and error methods to log.

Performance

  • Without parameters
    • 1000 iterations - 0.41s
    • 10,000 iterations - 3.98s
  • With four parameters
    • 1000 iterations - 0.95s
    • 10,000 iterations - 9.53s

Yay! :tada:! Reached the level of python native logger. Will be working on adding features and optimising code in a better way. :nerd_face:


Lo3 v0.2.0

I know, bad naming. But primary version of unoptimised logger written with Rust.

Primliminary scope

  • Adds log to file
  • Provides info, warn, debug and error methods to log.

Performance

  • 1000 iterations - 14.24s
  • 10,000 iterations - 161.64s

Not as expected, but have to explore opportunities in optimising rust code, as the current implementation is very crude.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lo3-0.3.0a0.tar.gz (7.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

lo3-0.3.0a0-cp312-cp312-macosx_11_0_arm64.whl (280.5 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

File details

Details for the file lo3-0.3.0a0.tar.gz.

File metadata

  • Download URL: lo3-0.3.0a0.tar.gz
  • Upload date:
  • Size: 7.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: maturin/1.7.1

File hashes

Hashes for lo3-0.3.0a0.tar.gz
Algorithm Hash digest
SHA256 edabc703622354ebb62adeebbf7059b3bb2aeb61c4b68d6a1b75083ae8978e86
MD5 90be2a5108f322e1f356e6abd8c6968e
BLAKE2b-256 9d58bde9d5b4c6491872612b51344598c977dce6c94ceb1ef0b695e362967f4c

See more details on using hashes here.

File details

Details for the file lo3-0.3.0a0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lo3-0.3.0a0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 efc824be498c929bd34e8dd4c7905b763ba051eb4843a71d335547abf675ccd1
MD5 40a4aebe0f49c092a79534fdae07169a
BLAKE2b-256 d95ce8b8a3edc557ccfff85dc0b7adc90f82ff36811f6ebb43971560d588c98a

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page