Skip to main content

No project description provided

Project description

Zarrita

Zarrita is an experimental implementation of Zarr v3 including sharding. This is only a technical proof of concept meant for generating sample datasets. Not recommended for production use.

Setup

import zarrita
import numpy as np

store = zarrita.LocalStore('testdata') # or zarrita.RemoteStore('s3://bucket/test')

Create an array

a = await zarrita.Array.create_async(
    store / 'array',
    shape=(6, 10),
    dtype='int32',
    chunk_shape=(2, 5),
    codecs=[zarrita.codecs.blosc_codec()],
    attributes={'question': 'life', 'answer': 42}
)
await a.async_[:, :].set(np.ones((6, 10), dtype='int32'))

Open an array

a = await zarrita.Array.open_async(store / 'array')
assert np.array_equal(await a.async_[:, :].get(), np.ones((6, 10), dtype='int32'))

Create an array with sharding

a = await zarrita.Array.create_async(
    store / 'sharding',
    shape=(16, 16),
    dtype='int32',
    chunk_shape=(16, 16),
    chunk_key_encoding=('v2', '.'),
    codecs=[
        zarrita.codecs.sharding_codec(
            chunk_shape=(8, 8),
            codecs=[zarrita.codecs.blosc_codec()]
        ),
    ],
)
data = np.arange(0, 16 * 16, dtype='int32').reshape((16, 16))
await a.async_[:, :].set(data)
assert np.array_equal(await a.async_[:, :].get(), data)

Create a group

g = await zarrita.Group.create_async(store / 'group')
g2 = await g.create_group_async('group2')
a = await g2.create_array_async(
    'array',
    shape=(16, 16),
    dtype='int32',
    chunk_shape=(16, 16),
)
await a.async_[:, :].set(np.arange(0, 16 * 16, dtype='int32').reshape((16, 16)))

Open a group

g = await zarrita.Group.open_async(store / 'group')
g2 = g['group2']
a = g['group2']['array']
assert np.array_equal(await a.asnyc_[:, :].get(), np.arange(0, 16 * 16, dtype='int32').reshape((16, 16)))

Credits

This is a largely-rewritten fork of zarrita by @alimanfoo. It implements the Zarr v3 draft specification created by @alimanfoo, @jstriebel, @jbms et al.

Licensed under MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zarrita-0.1.0a12.tar.gz (22.0 kB view hashes)

Uploaded Source

Built Distribution

zarrita-0.1.0a12-py3-none-any.whl (27.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page