fireworks-schema 1.4.2

JSON Schema for FireWorks

JSON Schema for FireWorks

This package provides a JSON schema for the FireWorks package.

Why should I use JSON schema?

The input for FireWorks is often provided in JSON or YAML formats and generated by third-party software that is unaware of the valid data types in FireWorks. Latent mismatches of data types may produce run-time errors, such as missing keywords or wrong data types, that are more difficult to handle than a validation of the initial input.

The fireworks_schema package provides a formal human- and machine-readable description of the data types used in classes in FireWorks. Additionally, a function is provided that checks the validity of JSON and YAML inputs immediately before deserialization.

Installing fireworks_schema

The recommended way to install this package into your virtual environment is using pip (in any folder):

python -m pip install fireworks-schema

Alternatively you can download a release from the GitHub repository, unpack the archive, change into the top-level folder containing setup.cfg and run:

python -m pip install .[test]

After the installation you can run the tests (from the top-level folder):

cd fireworks_schema/tests && pytest

Using fireworks_schema to validate input for FireWorks

There are two ways to perform JSON schema validation:

• Call the schema validator explicitly
• Activate automatic schema validation

Call the schema validator explicitly

This is the case when you use Python but read JSON/YAML serialized objects provided externally. In the following example, a serialized workflow object is loaded from a YAML file and validated against the Workflow schema:

  import yaml
  import fireworks_schema
  from fireworks import Workflow

  with open('empty_fws.yaml', 'rt') as yf:
      dct = yaml.safe_load(yf)
  fireworks_schema.validate(dct, 'Workflow')
  wf = Workflow.from_dict(dct)

Activate automatic schema validation

To activate automatic schema validation you must specify:

  JSON_SCHEMA_VALIDATE: true

in your FWConfig file. For more details about managing your FWConfig file see the FW Config tutorial.

The default value of JSON_SCHEMA_VALIDATE is false.

If automatic validation is turned on, i.e. JSON_SCHEMA_VALIDATE is true, then validation is performed only for the classes specified in the list JSON_SCHEMA_VALIDATE_LIST, whenever an object of these classes is loaded from file or from string. There is no default for JSON_SCHEMA_VALIDATE_LIST and therefore you must set JSON_SCHEMA_VALIDATE_LIST in your FWConfig file. For example, to turn on automatic validation for serialized Firework and Workflow objects these two lines must be added to the FWConfig file:

  JSON_SCHEMA_VALIDATE: true
  JSON_SCHEMA_VALIDATE_LIST: [Firework, Workflow]

FireWorks performs automatic validation only when from_file() or from_format() methods of the listed classes are called, i.e. only on reading from files and strings. This implies that the utility function load_object_from_file also performs validation. Otherwise FireWorks will not perform validation in scenarios such as dealing with documents from/to a database or from/to a URI. To validate after having read a document from a database / URI and before deserializing one should use fw_schema_deserialize to decorate the from_dict() method. To validate after serialization and before writing to a database / URI one should decorate the to_dict() method with fw_schema_serialize, e.g.:

from fireworks_schema import fw_schema_deserialize, fw_schema_serialize

class MyClass(FWSerializable):
    ...

    @classmethod
    @fw_schema_deserialize
    def from_dict(cls, dct):
        ...
        return cls(...)

    @fw_schema_serialize
    def to_dict(self):
        dct = ...
        return dct

Note that the decorators honor the JSON_SCHEMA_VALIDATE setting, which is False (default) or True, but not the JSON_SCHEMA_VALIDATE_LIST.

Exception handling

When a custom schema is not valid then SchemaError is raised. When an instance does not validate against a (valid) schema then ValidationError is raised. The local schema and instance are displayed after the error description. Nevertheless, the schema and the instance with which the validator has been called are not displayed. These are known if the schema validator is called explicitly, i.e. by calling fireworks_schema.validate(instance, schema_name). But when automatic validation is activated (either by the JSON_SCHEMA_VALIDATE_LIST or the decorator approaches outlined above) then these are hard to detect. For this purpose, the decorator accepts an optional debug parameter (False by default). By setting it to True the schema name and the instance are concatenated to the original message of the raised ValidationError.

Example:

class MyClass(FWSerializable):
    ...

    @classmethod
    @fw_schema_deserialize(debug=True)
    def from_dict(cls, dct):
        ...

Register external schemas

Schemas for all relevant classes in FireWorks are provided with this package and they work "out of the box". It is possible to use custom schemas for further classes, that are sub-classes of fireworks.utilities.fw_serializers.FWSerializable, provided by other packages but not FireWorks. For this, every schema has to be registered like this:

  import fireworks_schema
  fireworks_schema.register_schema('/absolute/path/to/the/schema.json')

After that, the schema can be used for both explicit and automatic validation as described in the previous sections.

Currently, these restrictions/conventions apply:

• The filename, e.g. schema.json as in the example, must be different from the file names of already registered schemas. This restriction comes about because the $id schema property is currently not used.
• With jsonschema < 4.18.0 all referenced schemas must be installed in the same directory. This implies that the default schemas provided with fireworks_schema cannot be used from custom schemas if jsonschema < 4.18.0.
• The schema filename is lower case of the schema name which in turn is the class name.
• Only JSON schema draft-07 must be used.

Here an example for a custom schema with filename fwintegerarray.json:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$ref": "#/FWIntegerArray",
  "FWIntegerArray": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "_fw_name": {
        "const": "{{FWIntegerArray}}"
      },
      "data": {
        "type": "array",
        "items": {
          "type": "integer"
        }
      }
    },
    "required": [
      "_fw_name"
    ]
  }
}