structured data
Project description
Structured data format (sdata)
Design goals
- open data format for open science projects
- self describing data
- flexible data structure layout
- hierarchical data structure (nesting groups, dictionaries)
- (posix path syntax support?)
- extendable data structure
- data format versions
- platform independent
- simple object model
- support of standard metadata formats (key/value, ...)
- support of standard dataset formats (hdf5, netcdf, csv, ...)
- support of standard dataset types (datacubes, tables, series, ...)
- support of physical units (conversion of units)
- transparent, optional data compression (zlib, blosc, ...)
- support of (de-)serialization of every dataset type (group, data, metadata)
- easy defineable (project) standards, e.g. for a uniaxial tension test (UT)
- (optional data encryption (gpg, ...))
- change management support?
- Enable use of data structures from existing tensor libraries transparently
- (single writer/ multiple reader (swmr) support)
- (nested data support)
Example data structure (brain storming)
└─mytestprogram
|
├─metadata.csv
|
├─testseries_c3c63f8094464325bd57623cb5bbe58f
| |
| ├─metadata.csv
| |
| ├─test_bb507e40663d49cca8264c0ed6751692
| | |
| | ├─metadata.csv
| | |
| | └─table_d061d7a58b2341128dd95412ff6ab36f
| | |
| | ├─metadata.csv
| | |
| | └─d061d7a58b2341128dd95412ff6ab36f.csv
| |
| └─test_e574e000f1404f5ebb9aaceb4183dc4c
| |
| └─metadata.csv
|
└─testseries_a6fc7decdb1441518f762e3b5d798ba7
|
├─metadata.csv
|
├─test_b62195ac49b64c9e8cedb7dba52bd539
| |
| ├─metadata.csv
| |
| ├─table_6322a66775604c32af74039575221fe0
| | |
| | ├─metadata.csv
| | |
| | └─6322a66775604c32af74039575221fe0.csv
| |
| └─table_9bec8c67e09b456f96a9e23b04d9441a
| |
| ├─metadata.csv
| |
| └─9bec8c67e09b456f96a9e23b04d9441a.csv
|
├─test_ddc82782f5f0455895145682fe0a70f2
| |
| ├─metadata.csv
| |
| └─table_ea4fa966bedb4104b9754a4f5f5d8a80
| |
| ├─metadata.csv
| |
| └─ea4fa966bedb4104b9754a4f5f5d8a80.csv
|
└─test_8796c35b2e3a4f8a82af181698c15861
|
├─metadata.csv
|
└─table_60a90898f0c94b23984b174a74a2a47a
|
├─metadata.csv
|
└─60a90898f0c94b23984b174a74a2a47a.csv
Metadata
Attributes
- name
- value
- dtype
- unit
- description
dtypes for attributes
- int
- float
- str
- timestamp (datetime.isoformat with timezone)
Each object has a metadata file.
Data model
The data model is orientated to the common data model, cp. https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/CDM/
Links
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
sdata-0.7.3.tar.gz
(27.6 kB
view hashes)