Skip to main content

A (de)serialization library for multidimensional arrays

Project description

seedee

A data format for n-dimensional arrays.

This project is in an early stage of development and should be used with caution

Installation

Python

Pre-built Python wheels can be installed via pip:

python3 -m pip install seedee

To build a wheel yourself, e.g., if your platform is not supported by the pre-built wheels, download the latest release and extract it. Inside the source directory, run:

$ cmake -S . -B build -DBUILD_PYTHON=ON -DCMAKE_BUILD_TYPE=Release
$ cmake --build build --target python-wheel --verbose

Building requires cmake version 3.13 or newer.

If everything works, the wheel is created in the build/python/dist/ subdirectory and can be installed via pip.

C/C++

To build and install the C/C++ libraries, run:

$ cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=path/to/install/dir
$ cmake --build build --target seedee --verbose
$ cmake --install build

Building requires cmake version 3.13 or newer.

Usage example

Python

import numpy as np
import seedee

# Create a numpy array of some dimension, shape, and dtype
array = np.arange(48).reshape((6, 8)).astype("uint16")

# Serialize the array using the bitshuffle+lz4 compression method
buffer, meta = seedee.serialize(array, seedee.COMPRESSION_BITSHUFFLE)
# buffer is a 1D numpy array of type uint8
# meta is a string

# Send buffer and meta to a remote process using any preferred method,
# e.g., via a ZMQ multi-message
...

# Deserialize the received buffer and meta into a numpy array
new_array = seedee.deserialize(buffer, meta)

# The resulting array is equal to the original array
assert np.all(array == new_array)

C/C++

See test directory for some examples. Note that the C/C++ interface will likely change in the future.

Limitations

Only arrays up to 1GB are supported.

Contribution

Please feel free to open issues or pull requests.

Acknowledgments

seedee depends on multiple open source libraries that are included in this repository under the thirdparty directory for convenience. The licenses of these dependencies can be found in the corresponding subdirectories and in LICENSE_THIRDPARTY.txt.

The chunk serializer implementations are based on the corresponding HDF5 filter plugins which are subject to their own licenses:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

seedee-0.3.0.tar.gz (12.4 kB view details)

Uploaded Source

Built Distributions

seedee-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (801.0 kB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

seedee-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (812.2 kB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

seedee-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (764.9 kB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

seedee-0.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767.9 kB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

seedee-0.3.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (741.7 kB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

seedee-0.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (684.3 kB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

seedee-0.3.0-cp36-cp36m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (675.2 kB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

File details

Details for the file seedee-0.3.0.tar.gz.

File metadata

  • Download URL: seedee-0.3.0.tar.gz
  • Upload date:
  • Size: 12.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.9.2

File hashes

Hashes for seedee-0.3.0.tar.gz
Algorithm Hash digest
SHA256 36779426dc35e587dc8f017ac5d3debf239e71cd078b698df133f3618dc1d0f1
MD5 d73f5be0360c20b7af44c9fe01b57f32
BLAKE2b-256 337ffb84c93c594cde130000e729a25206c3411d08402846bae50e6219da625d

See more details on using hashes here.

File details

Details for the file seedee-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for seedee-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8f8cdefad32b2a9ce343e693e77835d616bd2002e4dda41fecffb59267e45ee1
MD5 5ff91f832bdbc7c1e678dcd19e429b34
BLAKE2b-256 4dc48cbf816bfab692eb8f41e527deb183b828afb951f9cf8c125215d543c998

See more details on using hashes here.

File details

Details for the file seedee-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for seedee-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7df4db3f18d1c94049abe257924704308c971aba251fd1b665d2267daab3b005
MD5 817494b85e0feb98a2a531b02596c627
BLAKE2b-256 99629f210ab35006a2ce2c41f26f18a057a9eb020c51e21ee15e97680e2443ff

See more details on using hashes here.

File details

Details for the file seedee-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for seedee-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 46954a612aac0fb94726b37738eb1cfe4e74910cdebd96d9a4c099823a3ea7ec
MD5 9fb382be8a50b341798ed8bfb305fdd9
BLAKE2b-256 14998b9e3e3b2bd92450267fb715b82e7abd3e7199b950f330ba74625c69ec0f

See more details on using hashes here.

File details

Details for the file seedee-0.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for seedee-0.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 73445943489b589b53ca3476462a8ef2e29947bfd31a3168dd50225d20e3de72
MD5 127555d8a8093fce0c305d891acce5e5
BLAKE2b-256 dba9ef4f7b8e1c95a6dbaa6b567e4f0591b1f4e43d315c947411fd09bc4fbdd6

See more details on using hashes here.

File details

Details for the file seedee-0.3.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for seedee-0.3.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 3b1ba7338f797b8482038a431ed05753489b94dd709792c7887d4fad08c8e846
MD5 b642e92038301c6df1ad6bf1ccc7928b
BLAKE2b-256 32c15d98895ce2140d993fece5324194b948fc3caca1b41cc978341d40d49d71

See more details on using hashes here.

File details

Details for the file seedee-0.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for seedee-0.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 19a2ba2b0dd76f8278e6e735d2269fb4111e704aa11465e6e6e936fbe35649fa
MD5 986b5535ced032616fefc72d7c2fe571
BLAKE2b-256 27c6c05bdcc26f0db76a4697da677d68e956f85e7bc0966d26a01d429258499a

See more details on using hashes here.

File details

Details for the file seedee-0.3.0-cp36-cp36m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.

File metadata

File hashes

Hashes for seedee-0.3.0-cp36-cp36m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 9d65bfbf48d2ab53a6886f4ffaa16f4c90059f10335d2503e49072dea26e68e1
MD5 4a925ebb0d0eb8332b840aed7aff6a59
BLAKE2b-256 a57d7291503552ef2fb76693bacccffc9456c9c8eb991f03a083120ac398cece

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page