Skip to main content

Real time log viewer for flask applications

Project description

Flask Real Time Log Viewer

Real time log view

Screen Shot

Display logs in real time through a browser with flask.

Background

Looking at a few different projects and writes up showing how to stream logs to a browser and found they relied on WebSockets (wss), and Redis. Which complicates simple projects and increases expense. Flask / Python at large has a few hurdles

  • Websockets in gunicorns works, but only if you have 1 worker
    • This limits performance even with threads
  • uWSGI - maybe if you're willing to compile it
    • Dockerized slim images require the addition of gcc slowing the build down
  • Flasks Werkzeug isn't production hardened.

So the criteria for this project was

  • A simple web based real time log viewer
  • Could run across workers (forks)
  • Did not rely on WSS (Websockets)

This was designed using gunicorns but should work with any wsgi server.

Install & Configuration

This is a flask blueprint, and might not suit everyones setup but feel free to suggest changes.

pip install flask-log-viewer

Within your main flask app configure the following

from flask_log_viewer.viewer import log_viewer_blueprint
import os

log_directory = "/app/logs" #Directory logs are stored in
app = Flask(__name__)

log_viewer = log_viewer_blueprint(base_path=log_directory, allowed_directories=[log_directory])

app.register_blueprint(log_viewer, url_prefix='/logs')
......
......

This will now create the following urls

/logs/stream/<string: log_file>

Thus http://hostname/logs/stream/foo.log will look for a /app/logs/foo.log and begin tailing it emitting http stream events. Feel free to change the prefix as needed.

If you have logs in subdirectories like /app/logs/monday/foo.log

Using the included viewer

A reference viewer is included, using htmx to make it easier. This can be copied and modified to meet your needs and frameworks.

The viewer is at http://hostname/logs/view/ without a log name, a random set of strings is streamed back. Include a log file with http://hostname/logs/view/file-name

How logs are tailed

Under the covers we are using pygtail https://pypi.org/project/pygtail/ | https://github.com/bgreenlee/pygtail from Brad Greenlee. Pygtail is GPL licensed, use accordingly

Pygtail monitors log files for rotation, keeps track of the seek position, ensuring every time it's called, it goes back to the last position of the file being read from, which is stored on disk as 'file-name.log.offset'

This is useful in Server Side Events as sockets close and additional requests can be made by a client that can end up at any WSGI worker.

However it can also prevent a file from being re-read at the moment. Future //TODO: provide a method to reset

Files

What to look for if git cloning

├── LICENSE
├── README.md
├── app.py
├── app_logger
│   └── logger.py
├── flask_log_viewer
│   ├── static
│   │   └── js
│   │       └── ansi_html.js
│   ├── templates
│   │   └── viewer.html
│   └── viewer.py
├── poetry.lock
└── pyproject.toml
  • app.py sample flask app
  • app_logger - testing logger
  • flask_log_viewer
    • viewer.py blueprint code
    • templates/viewer.html - a htmx log viewer
    • static/js/ansi_html.js - basic ansi colors to html converter

Running with gunicorn

Using gevent with gunicorns reduces the amount of connect closes you receive and is recommended for production level hosting.

gunicorn -k gevent -w 4 -b 0.0.0.0:3001 --access-logfile - --error-logfile -  app:app 

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flask_log_viewer-0.2.2.tar.gz (6.3 kB view details)

Uploaded Source

Built Distribution

flask_log_viewer-0.2.2-py3-none-any.whl (7.2 kB view details)

Uploaded Python 3

File details

Details for the file flask_log_viewer-0.2.2.tar.gz.

File metadata

  • Download URL: flask_log_viewer-0.2.2.tar.gz
  • Upload date:
  • Size: 6.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.7

File hashes

Hashes for flask_log_viewer-0.2.2.tar.gz
Algorithm Hash digest
SHA256 f8d87d8cfa8a5dc5c4fefb25d13aec0924154a716efdc536a8ae10d0dbf106f0
MD5 4baef507d0dc436c89be4e3b676e079e
BLAKE2b-256 5246faeee74e9339a6e5de59a3ef50bda4ddd040cd2a46415120e855e358f9a6

See more details on using hashes here.

Provenance

File details

Details for the file flask_log_viewer-0.2.2-py3-none-any.whl.

File metadata

File hashes

Hashes for flask_log_viewer-0.2.2-py3-none-any.whl
Algorithm Hash digest
SHA256 6c2e6598ee5f26ade93d9b91c9beb928889bc3d36f260379647f246cc25e0e9c
MD5 5f685700efcd54ff7a6697b397e4e7d1
BLAKE2b-256 8c3723b6b171bfab39cdfaa76c3e5d02055d103c36d0066a49c5f7ce278260c1

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page