Skip to main content

Multiprocessing utils (shared locks and thread locals)

Project description

Multiprocessing utilities

Shared locks

“Shared” version of the standard Lock() and RLock() classes found in the multiprocessing/threading modules.

Shared locks can be acquired in two modes, shared and exclusive.

Any number of processes/threads can acquire the lock in shared mode.

Only one process/thread can acquire the lock in exclusive mode.

A process/thread attempting to exclusively acquire the lock will block until the lock has been released by all other threads/processes.

A process/thread attempting to shared acquire the lock will only block while there is an exclusive lock.

This is a little like database locks, which can be acquired for shared reading, or exclusive writing.

lock = multiprocessing_utils.SharedLock()

def exclusive_worker():
    with lock.exclusive():
        # this code will only run when no other
        # process/thread in a lock context

def shared_worker():
    with lock:
        # this code will run so long as no
        # thread/process holds an exclusive lock

multiprocess-safe threading.local()

A process (and thread) safe version of threading.local()

l = multiprocessing_utils.local()
l.x = 1

def f():
    try:
        print(l.x)
    except Attribute:
        print("x not set")
f()                                        # prints "1"
threading.Thread(target=f).start()         # prints "x not set"
multiprocessing.Process(target=f).start()  # prints "x not set"

Difference to standard threading.local()

A standard threading.local() instance created before forking (via os.fork() or multiprocessing.Process()) will be “local” as expected and the new process will have access to any data set before the fork.

Using a standard threading.local() in the example above would yield:

f()                                        # prints "1"
threading.Thread(target=f).start()         # prints "x not set"
multiprocessing.Process(target=f).start()  # prints "1" :(

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

multiprocessing_utils-0.4.tar.gz (6.7 kB view details)

Uploaded Source

Built Distribution

multiprocessing_utils-0.4-py2.py3-none-any.whl (10.5 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file multiprocessing_utils-0.4.tar.gz.

File metadata

File hashes

Hashes for multiprocessing_utils-0.4.tar.gz
Algorithm Hash digest
SHA256 43281d5e017d9b3f3e6114762c21f10c2a2b0392837c5096380e6c413ae79b6c
MD5 688bb388d0dbb506c37d11bccca2246b
BLAKE2b-256 7f23ee76f1f2d501411afc152208d30d1575a8e76929eca276b94f4e44600085

See more details on using hashes here.

File details

Details for the file multiprocessing_utils-0.4-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for multiprocessing_utils-0.4-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 c232e0bbc6ba753ca7a0df5d49b0cc4e26454635d4f373f5133c551aec8c27ee
MD5 7c057eb3f3378966d3a7e59868426718
BLAKE2b-256 705c2e3065be295ca10f39d22579fc59ba8dd258123b0d66209bb4ff7c3d4557

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page