Skip to main content

Cloud Array

cloud-array is an open-source Python library for storing and streaming large Numpy Arrays on local file systems and major cloud providers CDNs. It automatically chunks a large array of data into arbitrary chunks sizes and uploads them into the targeted direcotry.

import numpy as np
from cloud_array import CloudArray

shape = (10000, 100, 100)
chunk_shape = (10, 10, 10)

f = np.memmap(
   'memmapped.dat',
   dtype=np.float32,
   mode='w+',
   shape=shape
)

array = CloudArray(
   chunk_shape=chunk_shape,
   array=f,
   url="s3://example_bucket/dataset0"
)
array.save()
print(array[:100,:100,:100])

Metadata

Release files for cloud-array 0.0.7

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cloud-array 0.0.7
File Size Uploaded Provenance Actions
cloud_array-0.0.7.tar.gz 6.7 kB Without provenance Details for cloud_array-0.0.7.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for cloud-array 0.0.7
File Interpreter ABI Platform Actions
cloud_array-0.0.7-py3-none-any.whl Python 3 none any Details for cloud_array-0.0.7-py3-none-any.whl

Total release size: 13.8 kB

Release files / cloud_array-0.0.7.tar.gz

Download URL cloud_array-0.0.7.tar.gz
Size 6.7 kB
Tags Source
SHA-256 checksum
How to use checksums
0d6734c6a667051e210ca6bc34cd98211e59d91ce9d2e7f7aa794ce3680701d6
BLAKE2b-256 checksum
How to use checksums
7335a337e5fa7b4df43f7edb05770ef5fd489b87ff4525c7f1acfac6872a4a5f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.12

Release files / cloud_array-0.0.7-py3-none-any.whl

Download URL cloud_array-0.0.7-py3-none-any.whl
Size 7.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
59bf98f4ba30be6f28c7938dc8b91777a470321effbe50873cad7e37b98171af
BLAKE2b-256 checksum
How to use checksums
170e50522cece00e0542d07260ab556227c9af98b4573fa941a2de64b5b2a279
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.12

Security for cloud-array 0.0.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page