A polymorphic schema managed semi structured crosslinked data dictionary builder.. BINGO!
Project description
Datatables - structured data library based on schemas
Dev
poetry install
poetry shell
-- Make changes --poetry run pytest
poetry run black bstk_datatables
poetry run flake8 bstk_datatables
-- Commit & Push --
Install
pip install bstk-datatables
Overview
Datatables act as an intermediary between Marshmallow structures and user defined data storage structures.
It is designed to provide "just enough" sidechannel structure to facilitate building a dynamic schema, (and connecting with "other" interfaces), without losing the advantages afforded by static Marshmallow schemas.
Schema
Schema models are;
Schema
: A collection of fields and references that make up a partial or complete entrySchemaField
: A basic instruction container representing a single valueSchemaFieldFormat
: The specific instructions for how the field should be collected, represented, formatted and storedSchemaValuesError
: The only type of exception raised during schema validation
These schemas and fields are mapped to equivalent Marshmallow structures which provide the entry value validation mechanisms.. ref: Schema.process_values()
Entry
An Entry
is a collection of field values, references data, connector references and schema links.
.schemata
is a list ofSchema.code
's.table_id
is a link back to aTable.uuid
.references
and.connector_references
are unrestricted containers. Two containers are provided to seperate "core" references from "free-form" references..values
is a dict ofField.code
=>value
that conform to the listed schemata
Table
A Table
corrals one or more Entry
and shapes them towards one or more Schema
.
.schemata
is a list ofSchema.code
's that all entries must inherit.references
and.connectors
are unrestricted containers. Two containers are provided to seperate "core" references from "free-form" references (and allows correlation with table entries).
Marshalling and Persistence
All core classes (and Enum
) expose an export
method which return a dict.
The result of an export()
can be unpacked and provided to its constructor.
def test_entry_export():
data = {
"uuid": str(uuid4()),
"table_id": str(uuid4()),
"name": "Data Entry",
"references": {"entity_uuid": str(uuid4())},
"connector_references": {"connector1": "connector_ref"},
"schemata": ["base"],
"values": {"base/value1": "XG230"},
}
entry = Entry(**data)
exported = export(entry)
assert exported == data
The simplest way to handle data persistence is to encapsulate class instanciation and the export
method of the relevant class into an ORM or ODM framework.
MergeSchema
do not provide an export mechanism because they are not first-class citizens and are designed to work with established Schema
structures.
This test provides an example of how to implement persistence with flat files.
Extras
MergedSchema
Tables and Entries support more than a single schema reference.
MergedSchema
exists to facilitate mutli-schema validation and field ordering.
Provide Dict[Schema.Code: Schema]
as schemata
when initialising a MergedSchema
and it will:
- Process the schema in order
- De-dupe fields with the same code (If a later schema includes a field with the same code as a previously loaded schema - that field will be skipped)
- Provide a validation mechanism for entries
Enum
Enum are used within schemas as de-duped lookups. Multiple schema fields can use the same Enum for shaping values.
Usage:
- Provide an
Enum.code
as alookup
instead of avalues
list when supplyingSchemaFieldFormat
to a schemafield. - Provide the instanciated
Enum
toSchema.attach_lookup
on a compiledSchema
orMergedSchema
.
or
- Provide an instanciated
Enum
as alookup
instead of avalues
list when supplyingSchemaFieldFormat
to a schemafield.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for bstk_datatables-0.2.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fdc0aed20b594906a2d142dd44a4435abbcbd480c551aad74cfd9b158e726956 |
|
MD5 | 52b084112125ec2287da07f15cc1faf7 |
|
BLAKE2b-256 | 879f21c8ee491f421261f7e4b57fdb10ad2c1470f84fb9585f5519a76279bd7f |