NumPy Persistence to Non-Volatile Memory
Project description
NumPy Persistence to Non-Volatile Memory
This library will help you persist numpy array structures into NVM devices.
Internally, it uses pynvm
and buffers --which are
natively supported by numpy
.
Installation
pip install npp2nvm
Requirements
- A NVM device, such as Intel(R) Optane(TM) DC Persistent Memory modules.
- Python 3.x
- numpy arrays to be stored.
Usage
Two environment variables are used for configuring the storage:
- NPP2NVM_SIZE which specifies the size (in MiB) that will be assigned to persistence
- NPP2NVM_PATH which specifies the file path that will be used for persistence.
Note that this file should be located into a Storage Class Memory device, which will
allow
pynvm
to leverage its characteristics --keep in mind thatpynvm
are bindings of the PMDK libraries.
Example
import numpy as np
import npp2nvm
a = np.random.random([256, 256])
a = npp2nvm.np_persist(a)
print(a.shape)
# > (256, 256)
Limitations
Storage and retrieval of data structures (outside the application lifecycle) are not implemented in this library (yet?). You can implement your own mechanism and/or propose a PR. It will be appreciated!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
npP2NVM-0.2.dev2.tar.gz
(2.8 kB
view hashes)
Built Distribution
Close
Hashes for npP2NVM-0.2.dev2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | deb2f9beb101c70610a33019f94763354a5c8e22721fa880801a90f8714e9eeb |
|
MD5 | a9df215f4a5131b93bb930129b867ce1 |
|
BLAKE2b-256 | 812162225467ebab7e4a8b8f464fe6266a960e84a0f287640e2c67d06fe2f27c |