zarrita 0.1.0a1

No project description provided

Zarrita

Zarrita is an experimental implementation of Zarr v3 including sharding. This is only a technical proof of concept meant for generating sample datasets. Not recommended for production use.

Setup

import zarrita
import numpy as np

store = zarrita.FileSystemStore('file://./testdata')

Create an array

a = zarrita.Array.create(
    store,
    'array',
    shape=(6, 10),
    dtype='int32',
    chunk_shape=(2, 5),
    codecs=[zarrita.codecs.gzip_codec(level=1)],
    attributes={'question': 'life', 'answer': 42}
)
a[:, :] = np.ones((6, 10), dtype='int32')

Open an array

a = zarrita.Array.open(store, 'array')
assert np.array_equal(a[:, :], np.ones((6, 10), dtype='int32'))

Create an array with sharding

a = zarrita.Array.create(
    store,
    'sharding',
    shape=(16, 16),
    dtype='int32',
    chunk_shape=(16, 16),
    chunk_key_encoding=('v2', '.'),
    codecs=[
        zarrita.codecs.sharding_codec(
            chunk_shape=(8, 8),
            codecs=[zarrita.codecs.gzip_codec(level=1)]
        ),
    ],
)
data = np.arange(0, 16 * 16, dtype='int32').reshape((16, 16))
a[:, :] = data
assert np.array_equal(a[:, :], data)

Create a group

g = zarrita.Group.create(store, 'group')
g2 = g.create_group('group2')
a = g2.create_array(
    'array',
    shape=(16, 16),
    dtype='int32',
    chunk_shape=(16, 16),
)
a[:, :] = np.arange(0, 16 * 16, dtype='int32').reshape((16, 16))

Open a group

g = zarrita.Group.open(store, 'group')
g2 = g['group2']
a = g['group2/array']
assert np.array_equal(a[:, :], np.arange(0, 16 * 16, dtype='int32').reshape((16, 16)))

Credits

This is a largely-rewritten fork of zarrita by @alimanfoo. It implements the Zarr v3 draft specification created by @alimanfoo, @jstriebel, @jbms et al.

Licensed under MIT