Utilities for safely forking a process
Project description
safefork
Safefork is a small utility to check if your program is safe to be forked.
Motivation
In production environments, it’s common for web servers like Gunicorn and background task processors like Celery to offer a forking option. This forking approach can save significant memory and reduce startup times, but it can be difficult to audit if your application is safe to fork. It's even more difficult to maintain this safety as your application grows. Did a recent PR introduce a threaded metrics collector? Or maybe a threaded database connection pool?
Safefork tries to alleviate this problem by checking some common issues that can arise when forking. It ensures that there are no active threads, no pending signals, and no running event loops, and that the garbage collector has been configured to optimize memory savings.
Installation
pip install safefork
Example Usage: Flask + Gunicorn
## app.py
import gc
import flask
import safefork
# Disable GC until after we fork to avoid "memory holes".
gc.disable()
def create_app():
app = flask.Flask(__name__)
initialize_systems()
# Freeze an object generation to maximize shared memory across forked childs.
gc.freeze()
assert safefork.is_safe_to_fork(), "Uh-oh! The application is unsafe to fork"
return app
## gunicorn.conf.py
import gc
def post_fork(server, worker):
"""
Called just after a worker has been forked.
Enables garbage collection in the worker process.
"""
gc.enable()
server.log.info("Garbage collection enabled in worker process after fork.")
bind = '0.0.0.0:8000'
workers = 4
## Command to start application:
## gunicorn -c gunicorn.conf.py 'app:create_app()'
Contribution
Contributions are welcome! If you would like to contribute to this project, please include an example script in the unsafe_examples/ directory demonstrating how forking can be unsafe. This will help me and others understand potential issues. Appreciate your help in making this project better!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file safefork-1.0.0.tar.gz
.
File metadata
- Download URL: safefork-1.0.0.tar.gz
- Upload date:
- Size: 6.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: python-httpx/0.27.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7989e299b51e0825d8cabecd58b51ddb4a92bfa4da6543887a6c6ef38da8cbd5 |
|
MD5 | 26872696677bff4d0f60c2067b3e4287 |
|
BLAKE2b-256 | beaae0348b563891a6602312eed8b404c25d89d51295cee8cf40b9d7b4cfb6e4 |
File details
Details for the file safefork-1.0.0-py3-none-any.whl
.
File metadata
- Download URL: safefork-1.0.0-py3-none-any.whl
- Upload date:
- Size: 4.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: python-httpx/0.27.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5943744e4980e3d06ad9780907c85be327deafee34121dcd2ef62dea6117cc56 |
|
MD5 | db7c57bc9001b1273454edfde8fae5b2 |
|
BLAKE2b-256 | ecc12fb373f629f4d4d2fdfca4f241928824dd0c8c850a8e265eb02fc3722ae4 |