Skip to main content

Utilities for safely forking a process

Project description

safefork

Safefork is a small utility to check if your program is safe to be forked.

Motivation

In production environments, it’s common for web servers like Gunicorn and background task processors like Celery to offer a forking option. This forking approach can save significant memory and reduce startup times, but it can be difficult to audit if your application is safe to fork. It's even more difficult to maintain this safety as your application grows. Did a recent PR introduce a threaded metrics collector? Or maybe a threaded database connection pool?

Safefork tries to alleviate this problem by checking some common issues that can arise when forking. It ensures that there are no active threads, no pending signals, and no running event loops, and that the garbage collector has been configured to optimize memory savings.

Installation

pip install safefork

Example Usage: Flask + Gunicorn

## app.py
import gc
import flask
import safefork

# Disable GC until after we fork to avoid "memory holes".
gc.disable()

def create_app():
    app = flask.Flask(__name__)
    initialize_systems()
    # Freeze an object generation to maximize shared memory across forked childs.
    gc.freeze()
    assert safefork.is_safe_to_fork(), "Uh-oh! The application is unsafe to fork"
    return app

## gunicorn.conf.py
import gc

def post_fork(server, worker):
    """
    Called just after a worker has been forked.
    Enables garbage collection in the worker process.
    """
    gc.enable()
    server.log.info("Garbage collection enabled in worker process after fork.")

bind = '0.0.0.0:8000'
workers = 4


## Command to start application:
## gunicorn -c gunicorn.conf.py 'app:create_app()'

Contribution

Contributions are welcome! If you would like to contribute to this project, please include an example script in the unsafe_examples/ directory demonstrating how forking can be unsafe. This will help me and others understand potential issues. Appreciate your help in making this project better!

PyPI - Version PyPI - Python Version

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

safefork-1.0.0.tar.gz (6.1 kB view details)

Uploaded Source

Built Distribution

safefork-1.0.0-py3-none-any.whl (4.4 kB view details)

Uploaded Python 3

File details

Details for the file safefork-1.0.0.tar.gz.

File metadata

  • Download URL: safefork-1.0.0.tar.gz
  • Upload date:
  • Size: 6.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.27.0

File hashes

Hashes for safefork-1.0.0.tar.gz
Algorithm Hash digest
SHA256 7989e299b51e0825d8cabecd58b51ddb4a92bfa4da6543887a6c6ef38da8cbd5
MD5 26872696677bff4d0f60c2067b3e4287
BLAKE2b-256 beaae0348b563891a6602312eed8b404c25d89d51295cee8cf40b9d7b4cfb6e4

See more details on using hashes here.

File details

Details for the file safefork-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: safefork-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 4.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.27.0

File hashes

Hashes for safefork-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 5943744e4980e3d06ad9780907c85be327deafee34121dcd2ef62dea6117cc56
MD5 db7c57bc9001b1273454edfde8fae5b2
BLAKE2b-256 ecc12fb373f629f4d4d2fdfca4f241928824dd0c8c850a8e265eb02fc3722ae4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page