Darr is a Python science library for storing numeric data arrays in a format that is open, simple, and self-explanatory
Darr is a Python science library for storing and sharing numeric data arrays in a way that is open, simple, and self-explanatory. Save and use your numeric arrays and metadata with one line of code while long-term and tool-independent accessibility and easy shareability is ensured. In addition, Darr provides fast memory-mapped read/write access to such disk-based data and the ability to append data, , so that arrays may be larger than available RAM.
To maximize wide readability of your data, Darr is based on a combination of flat binary and human-readable text files. It automatically saves a description of how the data is stored, together with code for reading the specific data in a variety of current scientific data tools such as Python, R, Julia, IDL, Matlab, Maple, and Mathematica (see example array).
Darr is currently pre-1.0, still undergoing significant development.
- Purely based on flat binary and text files, tool independence.
- Supports very large data arrays through memory-mapped file access.
- Data read/write access through NumPy indexing
- Data is easily appendable.
- Human-readable explanation of how the binary data is stored is saved in a README text file.
- README also contains examples of how to read the array in popular analysis environments such as Python (without Darr), R, Julia, Octave/Matlab, GDL/IDL, Maple, and Mathematica.
- Many numeric types are supported: (u)int8-(u)int64, float16-float64, complex64, complex128.
- Easy use of metadata, stored in a separate JSON text file.
- Minimal dependencies, only NumPy.
- Integrates easily with the Dask or NumExpr libraries for numeric computation on very large Darr arrays.
See the documentation for more information.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.