Skip to main content

📖 A Python json-filelogger

Lint code and run tests pypi

A simple way to log data to json-files from within your code.

Installation

You can use the python package manager (pip) to install the file-logger:

pip install jsonfilelogger

Usage: LogWriter

Create a LogWriter:

from jsonfilelogger.logger import LogWriter

writer = LogWriter(folder=".", filename="log.json", threshold=10)

Parameters

folder: the folder where the logfile should be stored

filename: the name of the logfile

threshold: the threshold for auto-flushing; if set None the writer will not autoflush

Example

Log data to the logfile:

writer.log({"key_int": 1,
            "key_string": "logger",
            "key_boolean": True,
            "key_double": 1.337,
            "key_list": ["element1", "element2"],
            "key_none": None})

Usage: LogReader

Create a LogReader:

from jsonfilelogger.logger import LogReader

reader = LogReader(folder="./", filename="log.json")

Parameters

folder: the folder where the logfile should be stored

filename: the name of the logfile

Example

Read data from the logfile:

data = reader.retrieve()
... # do something with the data

Methods

Class Method Explanation
LogReader, LogWriter .reset() This method clears the contents of the logfile.
LogReader, LogWriter .remove() This methode removes the logfile.
LogWriter .with_datetime(folder: str, threshold: int) This method creates a logfile in the folder with the current timestamp as filename.
LogWriter .log(data) This method logs the data (dictionary) in the logfile.
LogWriter .flush() This method flushes the buffer in the writer. The threshold of the buffer is given in the constructor
LogReader .retrieve() This method retrieves all the data from the logfile and returns it as a list of dictionaries.

Demo usage

An example of how to use this logger is given. Imagine one has created a major breakthrough AI system that still has to be trained. During training one want's to keep an eye on the performance of the progress. To do this, the LogWriter can be added in the learning iterations (e.g. in one Jupyter notebook). In another process (e.g. in another Jupyter notebook) it is then possible to read in the data and make amazing visualisations (yay, visualisations!) of how your breakthrough model is performing...

from jsonfilelogger.logger import LogWriter

writer = LogWriter.with_datetime(folder="./", threshold=10)

super_great_ai_model = ...
iteration = 0
while iteration < 100_000:
    score, other_data = super_great_ai_model.do_iteration()
    writer.log({
        score: score,
        other_data: other_data
    })
    iteration += 1
writer.flush()
from jsonfilelogger.logger import LogReader
import glob

logs_files = sorted(glob.glob("./[0-9]*.json"))
reader = LogReader(folder="./", filename=logs_files[-1])

data = reader.retrieve()

# Make awesome visualisations!
...

There may also be other use cases, you can use this logger as you want!

Tests

You can run the tests yourselves by:

cd tests
pip3 install -r requirements.txt
pytest

License

This software is licensed under the MIT license.

Bugs/ Ideas?

In case of questions, ideas or bugs you can always create an issue or contact me.

Metadata

Release files for jsonfilelogger 1.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for jsonfilelogger 1.1.0
File Size Uploaded Provenance Actions
jsonfilelogger-1.1.0.tar.gz 5.4 kB Without provenance Details for jsonfilelogger-1.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for jsonfilelogger 1.1.0
File Interpreter ABI Platform Actions
jsonfilelogger-1.1.0-py3-none-any.whl Python 3 none any Details for jsonfilelogger-1.1.0-py3-none-any.whl

Total release size: 11.9 kB

Release files / jsonfilelogger-1.1.0.tar.gz

Download URL jsonfilelogger-1.1.0.tar.gz
Size 5.4 kB
Tags Source
SHA-256 checksum
How to use checksums
7ac66c63deec59010be5abb7dbda36ace3f012d0b3ee40d3820142af4dabf759
BLAKE2b-256 checksum
How to use checksums
a70a390b08a22ce94e95a45bfb9e9cb0b01af2ca14f1e9067a088f59c18515f9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

Release files / jsonfilelogger-1.1.0-py3-none-any.whl

Download URL jsonfilelogger-1.1.0-py3-none-any.whl
Size 6.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
f59e2bae7f0d62626f96d249fd4443c69156852870162b60a254734e1825a7f5
BLAKE2b-256 checksum
How to use checksums
7837bcf83f2715c40e4e425a2a312699c3675d60b3282999a7c07777fcf4428e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

Security for jsonfilelogger 1.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page