pyjeb 0.2.1

A powerfull lightweight library to check and variabilize your configuration files

Pyjeb

PyJeb is a powerfull lightweight library to check and variabilize your configuration files. The main features of pyjeb are:

• Control the structure of a configuration file (missing value, type, format, valid set)
• Add default value for missing fields
• Setup variable values (system or user defined)
• Allow case insensitive parameters
• Allow to add executable functions in configuration

Get started

Steps

1. Install PyJeb package
2. Setup control file
3. Setup configuration file
4. Call control_and_setup function

Install PyJeb

Install from PyPi package manager:

pip install pyjeb

Exemple

Setup a configuration file for your script.

In this exemple a yaml file will be used, but JSON can also be used.

Exemple of yaml configuration file - configuration.yaml

HR - Employees:
  source:
    path: "/Landing/HR/Employees/$sys.timestamp('YYYY-MM-DD')"
    pattern: "*.csv"
  target:
    path: "/Bronze/HR/Employees"
HR - Managers:
  source:
    path: "/Landing/HR/Managers/$sys.timestamp('YYYY-MM-DD')"
  target:
    path: "/Bronze/HR/Managers"
HR - Payroll:
  source:
    path: "/Landing/HR/Payroll/$sys.timestamp('YYYY-MM-DD')"
  target:
    path: "/Bronze/HR/Payroll"

Setup a configuration to describe the previous configuration file and add default value for non mendatory field

Exemple of yaml configuration file - control.yaml

control = [
  { "name": "source", "type": "dict" },
  { "name": "source.path" },
  { "name": "source.pattern", "default": "*" },
  { "name": "target", "type": "dict" },
  { "name": "target.path" },
]

Load the configuration and apply function control_and_setup function with control parameters.

This function will:

• Setup default if the value is not defined
• Setup value of variable ($sys.timestamp in this exemple)
• Set configuration as object (callable with dots)

import yaml
from pyjeb import control_and_setup

# load configuration file
with open("configuration.yaml") as f:
  configuration = yaml.load(f, Loader = yaml.loader.SafeLoader)

# control configuration
control = [
  { "name": "source", "type": "dict" },
  { "name": "source.path" },
  { "name": "source.pattern", "default": "*" },
  { "name": "target", "type": "dict" },
  { "name": "target.path" },
]

# loop on each item in configuration
for item in configuration:
  item_configuration = configuration[item]

  # apply the control and instantiate variables
  item_configuration = control_and_setup(item_configuration, control, to_object = True)

  # display values 
  print(f"--------------- {item}")
  print(f"source.path = '{item_configuration.source.path}'")
  print(f"source.pattern = '{item_configuration.source.pattern}'")
  print(f"target.path = '{item_configuration.target.path}'")

Output of the script

--------------- HR - Employees
source.path = '/Landing/HR/Employees/2023-12-14'
source.pattern = '*.csv'
target.path = '/Bronze/HR/Employees'
--------------- HR - Managers
source.path = '/Landing/HR/Managers/2023-12-14'
source.pattern = '*'
target.path = '/Bronze/HR/Managers'
--------------- HR - Payroll
source.path = '/Landing/HR/Payroll/2023-12-14'
source.pattern = '*'
target.path = '/Bronze/HR/Payroll'

setup_and_control function

The function control_and_setup is the only one to use in PyJeb. It use to apply controls and setup default and variables values.

Definition: control_and_setup( <configuration>, <controls>, [<variables>], [<functions>] )

Name	Mandatory	Type	Description
configuration	yes	Dict	A dictionary (of scalar or array or dictonnary) who contain the configuration to control and variabilize
controls	yes	List	A list of dictionnaries who define the structure of configuration. See all about the structure in controls page
variables	no	Dict	Custom variables accessible by configuration. See all about it in custom variable page
functions	no	Dict	Custom parametrized variables accessible by configuration. See all about it in custom variable page
to_object	no	Bool	Transform the output into an object. If false the output keeps dictionary format (default: false)