Skip to main content

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.

Project description

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.

SharedNDArrays are designed to be sent over multiprocessing.Pipe and Queue without serializing or transmitting the underlying ndarray or buffer. While the associated file descriptor is closed when the SharedNDArray is garbage collected, the underlying buffer is not released when the process ends: you must manually call the unlink() method from the last process to use it.

Usage

from __future__ import print_function

import multiprocessing as mp

import numpy as np
from shared_ndarray import SharedNDArray

try:
    shm = SharedNDArray((4, 4))
    shm.array[0, 0] = 1
    p = mp.Process(target=lambda shm: print(shm.array), args=(shm,))
    p.start()
    p.join()
finally:
    shm.unlink()

This should print:

[[ 1.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]

There are also convenience methods to create a new SharedNDArray from an existing NumPy array:

arr = np.array([0, 0])
shm1 = SharedNDArray.copy(arr)
shm2 = SharedNDArray.zeros_like(arr)
shm1.unlink()
shm2.unlink()

Dependencies

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shared-ndarray-1.1.1.tar.gz (3.3 kB view hashes)

Uploaded source

Built Distributions

shared_ndarray-1.1.1-py3-none-any.whl (6.0 kB view hashes)

Uploaded py3

shared_ndarray-1.1.1-py2-none-any.whl (6.0 kB view hashes)

Uploaded py2

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page