Skip to main content

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.

Project description

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.

SharedNDArrays are designed to be sent over multiprocessing.Pipe and Queue without serializing or transmitting the underlying ndarray or buffer. While the associated file descriptor is closed when the SharedNDArray is garbage collected, the underlying buffer is not released when the process ends: you must manually call the unlink() method from the last process to use it.

Usage

from __future__ import print_function

import multiprocessing as mp

import numpy as np
from shared_ndarray import SharedNDArray

try:
    shm = SharedNDArray((4, 4))
    shm.array[0, 0] = 1
    p = mp.Process(target=lambda shm: print(shm.array), args=(shm,))
    p.start()
    p.join()
finally:
    shm.unlink()

This should print:

[[ 1.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]

There are also convenience methods to create a new SharedNDArray from an existing NumPy array:

arr = np.array([0, 0])
shm1 = SharedNDArray.copy(arr)
shm2 = SharedNDArray.zeros_like(arr)
shm1.unlink()
shm2.unlink()

Dependencies

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for shared-ndarray, version 1.1.1
Filename, size File type Python version Upload date Hashes
Filename, size shared_ndarray-1.1.1-py2-none-any.whl (6.0 kB) File type Wheel Python version py2 Upload date Hashes View
Filename, size shared_ndarray-1.1.1-py3-none-any.whl (6.0 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size shared-ndarray-1.1.1.tar.gz (3.3 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page