📖 A Python json-filelogger
A simple way to log data to json-files from within your code.
Installation
You can use the python package manager (pip) to install the file-logger:
pip install jsonfilelogger
Usage: LogWriter
Create a LogWriter:
from jsonfilelogger.logger import LogWriter
writer = LogWriter(folder=".", filename="log.json", threshold=10)
Parameters
folder: the folder where the logfile should be stored
filename: the name of the logfile
threshold: the threshold for auto-flushing; if set None the writer will not autoflush
Example
Log data to the logfile:
writer.log({"key_int": 1,
"key_string": "logger",
"key_boolean": True,
"key_double": 1.337,
"key_list": ["element1", "element2"],
"key_none": None})
Usage: LogReader
Create a LogReader:
from jsonfilelogger.logger import LogReader
reader = LogReader(folder="./", filename="log.json")
Parameters
folder: the folder where the logfile should be stored
filename: the name of the logfile
Example
Read data from the logfile:
data = reader.retrieve()
... # do something with the data
Methods
| Class | Method | Explanation |
|---|---|---|
| LogReader, LogWriter | .reset() |
This method clears the contents of the logfile. |
| LogReader, LogWriter | .remove() |
This methode removes the logfile. |
| LogWriter | .with_datetime(folder: str, threshold: int) |
This method creates a logfile in the folder with the current timestamp as filename. |
| LogWriter | .log(data) |
This method logs the data (dictionary) in the logfile. |
| LogWriter | .flush() |
This method flushes the buffer in the writer. The threshold of the buffer is given in the constructor |
| LogReader | .retrieve() |
This method retrieves all the data from the logfile and returns it as a list of dictionaries. |
Demo usage
An example of how to use this logger is given. Imagine one has created a major breakthrough AI system that still has to be trained. During training one want's to keep an eye on the performance of the progress. To do this, the LogWriter can be added in the learning iterations (e.g. in one Jupyter notebook). In another process (e.g. in another Jupyter notebook) it is then possible to read in the data and make amazing visualisations (yay, visualisations!) of how your breakthrough model is performing...
from jsonfilelogger.logger import LogWriter
writer = LogWriter.with_datetime(folder="./", threshold=10)
super_great_ai_model = ...
iteration = 0
while iteration < 100_000:
score, other_data = super_great_ai_model.do_iteration()
writer.log({
score: score,
other_data: other_data
})
iteration += 1
writer.flush()
from jsonfilelogger.logger import LogReader
import glob
logs_files = sorted(glob.glob("./[0-9]*.json"))
reader = LogReader(folder="./", filename=logs_files[-1])
data = reader.retrieve()
# Make awesome visualisations!
...
There may also be other use cases, you can use this logger as you want!
Tests
You can run the tests yourselves by:
cd tests
pip3 install -r requirements.txt
pytest
License
This software is licensed under the MIT license.
Bugs/ Ideas?
In case of questions, ideas or bugs you can always create an issue or contact me.
Metadata
Release files for jsonfilelogger 1.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| jsonfilelogger-1.1.0.tar.gz | 5.4 kB | Without provenance | Details for jsonfilelogger-1.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| jsonfilelogger-1.1.0-py3-none-any.whl | Python 3 | none | any | Details for jsonfilelogger-1.1.0-py3-none-any.whl |
Total release size: 11.9 kB
Release files / jsonfilelogger-1.1.0.tar.gz
| Download URL | jsonfilelogger-1.1.0.tar.gz |
|---|---|
| Size | 5.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7ac66c63deec59010be5abb7dbda36ace3f012d0b3ee40d3820142af4dabf759
|
|
BLAKE2b-256 checksum How to use checksums |
a70a390b08a22ce94e95a45bfb9e9cb0b01af2ca14f1e9067a088f59c18515f9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
|
Release files / jsonfilelogger-1.1.0-py3-none-any.whl
| Download URL | jsonfilelogger-1.1.0-py3-none-any.whl |
|---|---|
| Size | 6.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f59e2bae7f0d62626f96d249fd4443c69156852870162b60a254734e1825a7f5
|
|
BLAKE2b-256 checksum How to use checksums |
7837bcf83f2715c40e4e425a2a312699c3675d60b3282999a7c07777fcf4428e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
|
Security for jsonfilelogger 1.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.