Cloud Array
cloud-array is an open-source Python library for storing and streaming large Numpy Arrays on local file systems and major cloud providers CDNs. It automatically chunks a large array of data into arbitrary chunks sizes and uploads them into the targeted direcotry.
import numpy as np
from cloud_array import CloudArray
shape = (10000, 100, 100)
chunk_shape = (10, 10, 10)
f = np.memmap(
'memmapped.dat',
dtype=np.float32,
mode='w+',
shape=shape
)
array = CloudArray(
chunk_shape=chunk_shape,
array=f,
url="s3://example_bucket/dataset0"
)
array.save()
print(array[:100,:100,:100])
Links
1 maintainer
Metadata
1 maintainer
Release files for cloud-array 0.0.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| cloud_array-0.0.7.tar.gz | 6.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| cloud_array-0.0.7-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 13.8 kB
Release files / cloud_array-0.0.7.tar.gz
| Download URL | cloud_array-0.0.7.tar.gz |
|---|---|
| Size | 6.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0d6734c6a667051e210ca6bc34cd98211e59d91ce9d2e7f7aa794ce3680701d6
|
|
BLAKE2b-256 checksum How to use checksums |
7335a337e5fa7b4df43f7edb05770ef5fd489b87ff4525c7f1acfac6872a4a5f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.10.12
|
Release files / cloud_array-0.0.7-py3-none-any.whl
| Download URL | cloud_array-0.0.7-py3-none-any.whl |
|---|---|
| Size | 7.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
59bf98f4ba30be6f28c7938dc8b91777a470321effbe50873cad7e37b98171af
|
|
BLAKE2b-256 checksum How to use checksums |
170e50522cece00e0542d07260ab556227c9af98b4573fa941a2de64b5b2a279
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.10.12
|
