An utility library that generates OARepo required data model files from a JSON specification file
Project description
OARepo model builder
An utility library that generates OARepo required data model files from a JSON specification file.
Installation
poetry install oarepo-model-builder
Data model specification file
Data model specification should be a JSON5 formatted file based on JSON Schema with the following OArepo specific extension keywords:
oarepo:type
- Defines a reference to a type of the given property/object.
- Can be used anywhere in the specification file.
- This directive will be replaced by a referred datamodel type contents.
- Only new properties will be added to current property/object.
- All referenced types must be registered under datamodel entrypoints (see Entrypoints)
Syntax
"oarepo:type": List[string] | string // list of datamodel type references or single string reference
Example Usage:
The following source specification:
// datamodel.json5
{
"title": "Test record v1.0.0",
"type": "object",
"additionalProperties": false,
"oarepo:type": [
"include1"
],
"properties": {
"field0": {
"oarepo:type": "include2"
}
}
}
// datamodels/include1.json5
{
"title": "Included properties v1.0.0",
"type": "object",
"additionalProperties": true,
"properties": {
"includedField1": {
"type": "string",
}
}
}
// datamodels/include2.json5
{
"type": "number"
}
# setup.py
entry_points = {
'oarepo_model_builder.datamodels': [
'includes = mypkg.datamodels'
],
...
}
Will be compiled into resulting JSON Schema:
{
"title": "Test record v1.0.0",
"type": "object",
"additionalProperties": false,
"properties": {
"field0": {
"type": "number"
},
"includedField1": {
"type": "string"
}
}
}
oarepo:search
- Specifies how the current field/object should be indexed and handled for search, filtering and aggregations
- Used by the ElasticSearch Mapping generator to generate property mappings.
- If not present on a property, a default mapping type of keyword or object (for object-type properties) is assumed in resulting mapping output
- Value can be either string or object.
- This directive is omitted from the JSON Schema builder output
Syntax
mapping_config: string // string value represents an ES mapping type of property
// or
mapping_config: object // you can also pass object for more complex property mapping configurations
// or
mapping_config: false // parent field should be omitted from the generated ES mapping output
"oarepo:search": {
"mapping": mapping_config // "oarepo:search" block will be substituted by mapping_object configuration ES mapping output
}
// or
"oarepo:search": string | false // "string" represents an ES mapping type of the parent property
Example usage
The following source specification:
{
"properties": {
"testNoMapping": {
"type": "string",
"oarepo:search": false
},
"testDefault": {
"type": "string"
},
"testExplicit": {
"type": "string",
"oarepo:search": {
"mapping": "text"
}
},
"testShorthand": {
"type": "string",
"oarepo:search": "date"
},
"testObject": {
"type": "string",
"oarepo:search": {
"mapping": {
"type": "text",
"index": "false"
}
}
},
"testArray": {
"type": "array",
"items": {
"type": "string",
"oarepo:search": "date"
}
}
}
}
Will result in the following files:
// mappings/v7/mymodel-v1.0.0.json
{
"mappings": {
"properties": {
"testDefault": {"type": "keyword"},
"testExplicit": {"type": "text"},
"testShorthand": {"type": "date"},
"testObject": {
"type": "text",
"index": "false"
},
"testArray": {"type": "date"}
}
}
}
// jsonschemas/.../mymodel-v1.0.0.json
{
"properties": {
"testNoMapping": {"type": "string"},
"testDefault": {"type": "string"},
"testExplicit": {"type": "string"},
"testShorthand": {"type": "string"},
"testObject": {"type": "string"},
"testArray": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
oarepo:ui
- Directive used to specify a field metadata to be used by UI representations of the data model
Syntax
multilingual_string: {lang_code: value, ...}
"oarepo:ui": {
"title": multilingual_string, // Property or object title
"label": multilingual_string, // Label to be displayed on property input fields
"hint": multilingual_string // Additional hint to be displayed on property input fields
}
Example usage
The following source specification:
{
"title": "Test record v1.0.0",
"type": "object",
"oarepo:ui": {
"title": {
"cs": "Testovaci zaznam",
"en": "Test record"
},
},
"properties": {
"field1": {
"type": "string",
"oarepo:ui": {
"label": {
"en": "Test field 1"
},
"hint": {
"cs": "Vyplnte testovaci field",
"en": "Please provide test field input"
}
}
}
}
}
Will result in the following files:
TODO(@mesemus):
Customization
Build configuration
You can override some build process defaults using a custom JSON configuration file, starting with configuration
from ./config/defauts.json
.
// build-config.json
{
"jsonschema": {
"type": "object",
"additionalProperties": false
},
"search": {
"default_mapping_type": "keyword",
"mapping": {
"settings": {
"analysis": {
"char_filter": {
"configured_html_strip": {
"type": "html_strip",
"escaped_tags": []
}
},
"normalizer": {
"wsnormalizer": {
"type": "custom",
"filter": [
"trim"
]
}
},
"filter": {
"czech_stop": {
"type": "stop",
"stopwords": "_czech_"
},
"czech_stemmer": {
"type": "stemmer",
"language": "czech"
}
},
"analyzer": {
"default": {
"tokenizer": "standard",
"filter": [
"lowercase",
"czech_stop",
"czech_stemmer"
]
}
}
}
},
"mappings": {
"dynamic": false,
"date_detection": false,
"numeric_detection": false,
"properties": {}
}
}
}
}
Entrypoints
This package uses the following entrypoints in the build process to determine, which builders and data models should be considered:
oarepo_model_builder.source
Classes responsible for parsing the whole source data model specification file.
Default:
datamodel = "oarepo_model_builder.handlers:DataModelBuilder"
oarepo_model_builder.elements
Classes for building the output files from elements in a source data model specification file.
Default:
jsonschema = "oarepo_model_builder.builders.jsonschema_builder:JSONSchemaBuilder"
mapping = "oarepo_model_builder.builders.mapping:MappingBuilder"
oarepo_model_builder.{output_type}
Classes responsible for generating output files of certain type
Default:
[tool.poetry.plugins."oarepo_model_builder.jsonschema"]
jsonschema = "oarepo_model_builder.outputs:JsonSchemaOutput"
[tool.poetry.plugins."oarepo_model_builder.mapping"]
mapping = "oarepo_model_builder.outputs:MappingOutput"
oarepo_model_builder.datamodels
Python modules containing data model specification files
Usage
To build a data model files from a specification file, this package provides the models
script:
$ models build --help
Usage: models build [OPTIONS] SOURCE
Build data model files from JSON5 source specification.
Options:
--package TEXT Package name of the model (example: 'test-package')
--config PATH Path to custom build config file (example: './build-config.json')
--datamodel-version TEXT Version string of the built model: (example: '1.0.0')
--help
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for oarepo-model-builder-0.1.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d427b1cddc8a3e26ac885a3b08c205d776ef72e72e3727ce00f69490fd550473 |
|
MD5 | e9ce61708963b4f759e5d7fcb0ab7c51 |
|
BLAKE2b-256 | a1185345c25fa01686d9407ca29fb4e0484180b2d261680475013b43f2247373 |
Hashes for oarepo_model_builder-0.1.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 71aede0199a88088d6aae4cc579cc7efc9c475de46fc3473bc3f69ed5a7ba181 |
|
MD5 | c51756ca75374e7d0374ce88823fae60 |
|
BLAKE2b-256 | e68274516658a9440b4fa1c0d42ed2a5bfd40bcbbfab91e708c22f989b91a479 |