specker 1.2.1

Specker JSON Specification Validator

Specker

The JSON Configuration Validator

• Specker
  • About
    • How does it work?
  • Usage
    • Spec Files
    • Validation
  • Examples
    • Example Data
      • Example Spec: myfile.json.spec
      • Example Spec: myfile.json_time.spec
      • Example Spec: myfile.json_env.spec
      • Example Spec: myfile.json_env.deep.spec
    • Example - Validation without Chaining
    • Example - Validation with Chaining
    • Example - Deep Chain Validation
  • Utils

About

Specker is a way to validate a configuration (Dictonary, or JSON) against a defined set of rules, while also providing defaults. Additionally, because the configuration is now documented as one of the Spec files, documentation for the specific configuration can be generated from the Spec file!

How does it work?

Specker uses a dictionary block for each specified parameter in order to validate that it is the correct type, and that it exists, if required. If it is not a required value, a default can also be set. Spec files contain a defined group or configuration file, for example, if we wanted to validate myconfig.json, we would create a Spec called myconfig.json.spec. This Spec would be loaded, and then the contents of myconfig.json would be compared against it.

Usage

Spec Files

See SPECFILES.md for information on the required values for each entry in a Spec. Spec files must be saved as a .spec file. For speed, specs should be kept in their own directory, so no other files need to be searched through.

Spec Files are made up of many blocks of Spec rules which define what a configuration block must look like. Specker is even capable of self-specing itself! You can see the Spec file for all other Spec files by examining the specs/specker.spec file.

Validation

Using Specker is easy! Once you've made your Spec file(s), you only need to load Specker, and your configuration, then compare the two!

SpeckerLoader.compare() will return a boolean pass/fail. Failure will occur if any of the spec validation fails. Validation messages are logged in via logging.*. This includes having values that are not in the spec file.

### Load Your Configuration
import json
try:
    with open("myfile.json", "r", encoding="utf-8") as f:
        config_content = json.loads(f.read())
except BaseException as e:
    print(f"Failed to load myfile, {e}")
    exit(1)

### Load and Use Specker
# Import the Specker Loader
from specker.loader import SpeckerLoader

# Initialize the Loader, and point it to your Spec directory
spec_loader = SpecLoader("../specs/") # Load all .spec files from this directory

Specker by default will only validate the dictionary you pass, and no further dictionaries or lists within it. By adding spec_chains, Specker can traverse deeper into the tree.

The Example below describes a 'root' level spec, as well as a 'sub' level spec, and how to validate a configuration against them.

Examples

Example Data

Example Spec: myfile.json.spec

{
    "name": {
        "required": true,
        "default": "",
        "type": "str",
        "comment": "Job Name (Identifier)",
        "example": "myexample"
    },
    "time": {
        "required": true,
        "default": {},
        "type": "dict",
        "comment": "Time Configuration"
    },
    "environment": {
        "required": false,
        "default": {},
        "type": "dict",
        "comment": "Environment Variables to use during Command execution"
    },
    "visibility": {
        "required": false,
        "default": "hidden",
        "type": "str",
        "values": [ "hidden", "visible", "admin", "deleted" ],
        "comment": "Visbility of Command Information"
    }
}

Example Spec: myfile.json_time.spec

{
    "minute": {
        "required": true,
        "default": "",
        "type": "str",
        "comment": "Minute(s) to Run at",
        "example": "*/5"
    },
    "hour": {
        "required": true,
        "default": "",
        "type": "str",
        "comment": "Hour(s) to Run at",
        "example": "*/2"
    },
    "day-of-month": {
        "required": true,
        "default": "",
        "type": "str",
        "comment": "Day(s) of Month to Run at",
        "example": "*"
    },
    "month": {
        "required": true,
        "default": "",
        "type": "str",
        "comment": "Month(s) to Run at",
        "example": "*"
    },
    "day-of-week": {
        "required": true,
        "default": "",
        "type": "str",
        "comment": "Day(s) of Week to Run at",
        "example": "1"
    }
}

Example Spec: myfile.json_env.spec

{
    "my_special_var": {
        "type": "str",
        "required": true,
        "default": "",
        "comment": "My Special Variable",
        "example": "a_cool_thing"
    },
    "my_deep_validation": {
        "type": "dict",
        "required": true,
        "default": {},
        "comment": "A Deeper dictionary",
        "spec_chain": "myfile.json_env.deep"
    }
}

Example Spec: myfile.json_env.deep.spec

{
    "my_other_var": {
        "type": "str",
        "required": true,
        "default": "not_set",
        "comment": "Some other deep level variable",
        "values": [ "a", "b", "foo", "bar", "not_set" ]
    }
}

Example - Validation without Chaining

Now that we have our Specs, we can compare them against our configuration data.

# Initialize the Loader, and point it to your Spec directory
spec_loader = SpecLoader("../specs/") # Load all .spec files from this directory

# Load `myfile.json.spec` and compare `config_content` against it.
spec_result = spec_loader.compare("myfile.json",config_content)
if not spec_result:
    exit(1)
# Because we are not chaining specs together, the `time` dictionary must be validated separately
spec_result = spec_loader.compare("myfile.json_time",config_content["time"])
if not spec_result:
    exit(1)
exit(0)

Example - Validation with Chaining

Before we compare our specs, we will make one modification to myspec.json.spec. Within the time definition, we will add "spec_chain": "myspec.json_time".

{
    "name": {
        ...
    },
    "time": {
        "required": true,
        "default": {},
        "type": "dict",
        "comment": "Time Configuration",
        "spec_chain": "myspec.json_time"
    },
    "environment": {
        ...
    },
    ....
}

Now that we have our Specs, we can compare them against our configuration file:

# Initialize the Loader, and point it to your Spec directory
spec_loader = SpecLoader("../specs/") # Load all .spec files from this directory

# Load `myfile.json.spec` and compare `config_content` against it.
spec_result = spec_loader.compare("myfile.json",config_content)
if not spec_result:
    exit(1)
exit(0)

Example - Deep Chain Validation

If we additionally change the environment definition in myspec.json.spec to add "spec_chain": "myspec.json_env", we can show deeper chained validation.

In this case, myspec.json.spec will be the starting point. Upon reaching the environment definition, it will utilize the myspec.json_env.spec spec to validate the environment dictionary. Upon reaching the my_deep_validation definition, it will utilize the myspec_json_env.deep.spec spec to validate the my_deep_validation dictionary.

No code is shown, as this case would be exactly the same code as the validation with chaining example

Utils

To aid in documentation, Specker comes with generate-spec-docs.py, a script to generate a .md file from a directory of Spec files. (See SPECFILES.md, this is a generated document).

You can additionally validate a Spec file using validate-spec-file.py, to ensure that your Spec file is built properly.