Skip to main content

Tool for reading and writing datasets of tensors with MessagePack and Lightning Memory-Mapped Database (LMDB)

Project description


Tool for reading and writing datasets of tensors (numpy.ndarray) with MessagePack and Lightning Memory-Mapped Database (LMDB).


import numpy as np
import pyxis as px

# Create data
nb_samples = 10
X = np.ones((nb_samples, 2, 2), dtype=np.float32)
y = np.arange(nb_samples, dtype=np.uint8)

# Write
db = px.Writer(dirpath='data', map_size_limit=1)
db.put_samples('input', X, 'target', y)

# Read
db = px.Reader(dirpath='data')
sample = db[0]

{'input': array([[ 1.,  1.], [ 1.,  1.]], dtype=float32), 'target': array(0, dtype=uint8)}

More examples can be found in the examples/ directory.


The installation instructions are generic and should work on most operating systems that support the prerequisites.

ml-pyxis requires Python version 2.7, 3.4, 3.5, or 3.6. We recommend installing ml-pyxis, as well as all prerequisites, in a virtual environment via virtualenv.


The following Python packages are required to use ml-pyxis:

Please refer to the individual packages for more information about additional dependencies and how to install them for your operating system.

Bleeding-edge installation

To install the latest version of ml-pyxis, use the following command:

pip install --upgrade

Add the --user tag if you want to install the package in your home directory.


The previous LMDB-only API has been deprecated in favour of a combination between LMDB and msgpack. The old version can be installed by using the following commit hash with pip:

pip install --upgrade git+git://

Development installation

ml-pyxis can be installed from source in such a way that any changes to your local copy will take effect without having to reinstall the package. Start by making a copy of the repository:

git clone

Next, enter the directory and install ml-pyxis in development mode by issuing the following command:

cd ml-pyxis
python develop

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ml-pyxis-0.4.dev0.tar.gz (11.2 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page