offlog 0.1.0

A non-blocking file-append service for Unix systems

offlog 0.1

offlog is a non-blocking file-append service for Unix systems. It consists of a server program and Python client library that interact via a Unix domain socket, allowing applications to offload file appends to the server and mitigating the risk of application latency caused by blocking file I/O. It has a focus on performance, specifically on minimising overhead for the client.

Both client and server are written in Python, but the protocol is not Python-specific so other clients could be written as well. This package has no dependencies outside the Python standard library.

example

Example:

# example client code example.py
from offlog import Logger
logger = Logger("my app", '/tmp/myapp.log')
logger.info("Hello world, params are %s %d", "foo", 7)
try:
    1/0
except Exception:
    logger.exception("Got an error")

Start the the server in one terminal, and then run the client in another. You'll see the following:

$ python -m offlog
[2023-07-03 10:12:21.895 offlog INFO] This is offlog server
[2023-07-03 10:12:21.895 offlog INFO] Listening on socket /tmp/offlog.sock
[2023-07-03 10:12:54.796 offlog INFO] Client 0 connected
[2023-07-03 10:12:54.796 offlog INFO] Client 0 access confirmed for /tmp/myapp.log
[2023-07-03 10:12:54.796 offlog INFO] New client 0 (total: 1) for /tmp/myapp.log
[2023-07-03 10:12:54.799 offlog INFO] Client 0 disconnected
[2023-07-03 10:12:54.799 offlog INFO] Client 0 done (remaining: 0) with /tmp/myapp.log
[2023-07-03 10:12:54.799 offlog INFO] Closed /tmp/myapp.log

$ python example.py
[2023-07-03 10:12:54.796 my app INFO] Hello world, params are foo 7
[2023-07-03 10:12:54.796 my app ERROR] Got an error
Traceback (most recent call last):
  File "/path/to/example.py", line 6, in <module>
    1/0
    ~^~
ZeroDivisionError: division by zero

And the contents of /tmp/myapp.log will be:

$ cat /tmp/myapp.log
[2023-07-03 10:12:54.796 my app INFO] Hello world, params are foo 7
[2023-07-03 10:12:54.796 my app ERROR] Got an error
Traceback (most recent call last):
  File "/path/to/example.py", line 6, in <module>
    1/0
    ~^~
ZeroDivisionError: division by zero

Details and notes

The Logger() class by default outputs DEBUG and higher to file, INFO to stdout, and ERROR and higher to stderr. These are configurable. Formatting and exception options are similar to the Python standard library.

The focus of this library is more on low-overhead file appends than the Logger() class, which is very basic. It is easily subclassable if you want to change some of its behaviour, or you could feasibly plug the FileProxy() object that sends data to the offlog server into some other logging framework. You may also use the FileProxy object directly to proxy file appends, including of binary data, to a file.

Despite being offloaded to another process, all file appends are still reliable, in the sense that all data will eventually be written if the server and client shut down in normal ways. This includes if the client encounters a Python exception, but does not include if the client, e.g. segfaults or is terminated via SIGTERM and Python's default SIGTERM handler (which exits immediately without doing any normal Python cleanup).

The server does flush files after every write, so when things are running normally, logs are written in real-time. The only reason they might be delayed is when the system is under load and messages are getting backed up - in which case clients will not retry sending queued data until the next time they have new data to send, or until program shutdown, whichever comes first.

There is no facility for global look up of loggers. Pass them around yourself. They are not thread-safe, but if you instantiate one per thread pointing to the same file, they will both talk to server which will interleave their appends. But their data won't be guaranteed to be in the same order or even have the same boundaries as they were fired off in your application.

Instantiating a FileProxy is blocking - the server will verify it can open the file and we wait for it to confirm. Writing data is not ever blocking, and for now data that fails to send because the server is too busy will just be locally buffered indefinitely (you can still run out of memory). Unsent data will be retried before sending new data on subsequent calls to write(). At shutdown, all unsent data will attempted to be flushed to the server, this is also blocking.

If the server is shut down or encounters an exception writing data on the client's behalf, it will terminate the connection with the client. Upon the next attempt to write the client will get a BrokenPipeError and read the exception message from the server, raising it as an exception in the client code.

Client documentation

Logger

offlog.Logger(
    name=None,
    filepath=None,
    file_level=DEBUG,
    stdout_level=INFO,
    stderr_level=WARNING,
    local_file=False,
    offlog_socket_path='/tmp/offlog.sock',
    offlog_timeout=5000
)

Logging object to log to file, stdout and stderr, with optional proxying of file writes via a offlog server.

Records with level file_level and above will be written to the given file. Records with level stdout_level and above, up to but not including stderr_level will be written to stdout, or with no upper limit if stdout_level is None. Records with level stderr_level and above will be written to stderr. Any of these can be set to None to disaable writing to that stream. filepath=None will also disable file logging.

if local_file is True, then an ordinary file will be opened for writing. Otherwise offlog_socket_path is used to connect to a running offlog server, which will open the file for us, writes will be proxied through it. Any blocking operations communicating with the server (such as the initial file open, and flushing data at shutdown) will be subject to a communications timeout of offlog_timeout in milliseconds, default 5000.

UTF-8 encoding is assumed throughout.

offlog.Logger.close()

Close the file. Possibly blocking. Idempotent.

Logging methods work similarly to the Python standard library:

offlog.Logger.log(level, msg, *args, exc_info=False):

offlog.Logger.debug(msg, *args, exc_info=False):

offlog.Logger.info(msg, *args, exc_info=False):

offlog.Logger.warning(msg, *args, exc_info=False):

offlog.Logger.error(msg, *args, exc_info=False):

offlog.Logger.exception(msg, *args, exc_info=True)

offlog.Logger.critical(msg, *args, exc_info=False)

ProxyFile

offlog.ProxyFile(
    filepath,
    sock_path='/tmp/offlog.sock',
    timeout=5000,
)

Object to proxy appending file writes via a running offlog server.

offlog.ProxyFile.write( data):

Send as much as we can without blocking. If sending would block, queue unsent data for later. On BrokenPipeError, check if the server sent us an error and raise it if so. This method will always attempt to re-send previously-queued data before attempting to send new data.

offlog.ProxyFile.close():

Close the socket. Attempt to send all queued unsent data to the server and cleanly close the connection to it, raising exceptions if anything goes wrong

Server documentation

$ python -m offlog -h
usage: python -m offlog [-h] [-n] [-s SOCKET_FILE] [-l LOGFILE]

offlog file-append service

options:
  -h, --help            show this help message and exit
  -n, --notify          Notify systemd when the server has started up and is
                        listening.
  -s SOCKET_FILE, --socket-file SOCKET_FILE
                        Path of the Unix socket the server binds to. Default:
                        /tmp/offlog.sock
  -l LOGFILE, --logfile LOGFILE
                        Path of the (optional) log file for the offlog server
                        itself.

You may wish to start the service as a systemd service. Here is an example unit file:

# offlog.service
[Unit]
Description=offlog file-append service
After=network.target

[Service]
Type=notify
ExecStart=/path/to/your/.venv/bin/python -u -m offlog --notify
Restart=always
RestartSec=5s
User=<user_to_run_as>

[Install]
WantedBy=multi-user.target

Note: the server will run with permissions of the specific user, which means all file operations will be performed as that user. The server's unix socket file will be created with default access permissions of that user, and thus only users will permission to access that socket will be able to communicate with the server. This means by default, clients will not be able to access an offlog server running as root in order to write to files they would not have otherwise had permission to write to.

To avoid race conditions in systemd starting other services that depend on the offlog server, ensure you use the --notify flag to the offlog server, which will notify systemd when the server's socket is bound and it is ready to accept clients. Mark other units that require the offlog server by adding Requires and After lines to the [Unit] section of their service files:

[Unit]
Description=Some other service that needs an offlog server
...
Requires=offlog.service
After=offlog.service

protocol

TODO - document the protocol