Skip to main content

Darr is a Python science library for storing numeric data arrays in a format that is open, simple, and self-explanatory

Project description

Darr is a Python science library for storing and sharing numeric data arrays in a way that is open, simple, and self-explanatory. Save and use your numeric arrays and metadata with one line of code while long-term and tool-independent accessibility and easy shareability is ensured. In addition, Darr provides fast memory-mapped read/write access to such disk-based data and the ability to append data, , so that arrays may be larger than available RAM.

To maximize wide readability of your data, Darr is based on a combination of flat binary and human-readable text files. It automatically saves a description of how the data is stored, together with code for reading the specific data in a variety of current scientific data tools such as Python, R, Julia, IDL, Matlab, Maple, and Mathematica (see example array).

Darr is currently pre-1.0, still undergoing significant development.

Features

  • Purely based on flat binary and text files, tool independence.
  • Supports very large data arrays through memory-mapped file access.
  • Data read/write access through NumPy indexing
  • Data is easily appendable.
  • Human-readable explanation of how the binary data is stored is saved in a README text file.
  • README also contains examples of how to read the array in popular analysis environments such as Python (without Darr), R, Julia, Octave/Matlab, GDL/IDL, Maple, and Mathematica.
  • Many numeric types are supported: (u)int8-(u)int64, float16-float64, complex64, complex128.
  • Easy use of metadata, stored in a separate JSON text file.
  • Minimal dependencies, only NumPy.
  • Integrates easily with the Dask or NumExpr libraries for numeric computation on very large Darr arrays.

See the documentation for more information.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

darr-0.1.10.tar.gz (45.7 kB view hashes)

Uploaded source

Built Distribution

darr-0.1.10-py3-none-any.whl (40.0 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page