Skip to main content

Provide shared memory data structures!

Project description

Shared Memory Data Structure

This package allows you to use your data structures like numpy arrays in the shared memory environment between two or more python processes. This library simplifies the use of shared memory data structures as you don't need to manually manage shared memory.

SharedArray example:

Process #1

from shared_ds import SharedArray

# Create shared memory and put you numpy array into that memory segment.
shared_np_array = SharedArray.from_array(np_array)

shm_descriptor = shared_np_array.to_json()

Process #2

from shared_ds import SharedArray

# Attaches to existing shared memory and reads numpy array representation.
shared_np_array = SharedArray.from_json(shm_descriptor)

shm_descriptor = shared_np_array.to_json()

io.BytesIO example:

Process #1

from shared_ds import SharedBytesIO
import io

# Create shared memory and put content of passed BytesIO into that memory segment.
data_to_store = io.BytesIO(b'data which we want to store')
shared_memory = SharedBytesIO.from_bytes_io(data_to_store)

shm_descriptor = shared_memory.to_json()

Process #2

from shared_ds import SharedBytesIO

# Attaches to existing shared memory and gets io.BytesIO content.
shared_memory = SharedBytesIO.from_json(shm_descriptor)

data = shared_memory.get_data()

Currently supported data structures:

  • Numpy Array
  • io.BytesIO

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shared_ds-0.0.5.tar.gz (3.4 kB view hashes)

Uploaded source

Built Distribution

shared_ds-0.0.5-py2.py3-none-any.whl (4.5 kB view hashes)

Uploaded py2 py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page