A powerfull lightweight library to check and variabilize your configuration files
Project description
Pyjeb
PyJeb is a powerfull lightweight library to check and variabilize your configuration files. The main features of pyjeb are:
- Control the structure of a configuration file (missing value, type, format, valid set)
- Add default value for missing fields
- Setup variable values (system or user defined)
- Allow case insensitive parameters
- Allow to add executable functions in configuration
Get started
Steps
- Install PyJeb package
- Setup control file
- Setup configuration file
- Call control_and_setup function
Install PyJeb
Install from PyPi package manager:
pip install pyjeb
Exemple
Setup a configuration file for your script.
In this exemple a yaml file will be used, but JSON can also be used.
Exemple of yaml configuration file - configuration.yaml
HR - Employees:
source:
path: "/Landing/HR/Employees/$sys.timestamp('YYYY-MM-DD')"
pattern: "*.csv"
target:
path: "/Bronze/HR/Employees"
HR - Managers:
source:
path: "/Landing/HR/Managers/$sys.timestamp('YYYY-MM-DD')"
target:
path: "/Bronze/HR/Managers"
HR - Payroll:
source:
path: "/Landing/HR/Payroll/$sys.timestamp('YYYY-MM-DD')"
target:
path: "/Bronze/HR/Payroll"
Setup a configuration to describe the previous configuration file and add default value for non mendatory field
Exemple of yaml configuration file - control.yaml
control = [
{ "name": "source", "type": "dict" },
{ "name": "source.path" },
{ "name": "source.pattern", "default": "*" },
{ "name": "target", "type": "dict" },
{ "name": "target.path" },
]
Load the configuration and apply function control_and_setup function with control parameters.
This function will:
- Setup default if the value is not defined
- Setup value of variable ($sys.timestamp in this exemple)
- Set configuration as object (callable with dots)
import yaml
from pyjeb import control_and_setup
# load configuration file
with open("configuration.yaml") as f:
configuration = yaml.load(f, Loader = yaml.loader.SafeLoader)
# control configuration
control = [
{ "name": "source", "type": "dict" },
{ "name": "source.path" },
{ "name": "source.pattern", "default": "*" },
{ "name": "target", "type": "dict" },
{ "name": "target.path" },
]
# loop on each item in configuration
for item in configuration:
item_configuration = configuration[item]
# apply the control and instantiate variables
item_configuration = control_and_setup(item_configuration, control, to_object = True)
# display values
print(f"--------------- {item}")
print(f"source.path = '{item_configuration.source.path}'")
print(f"source.pattern = '{item_configuration.source.pattern}'")
print(f"target.path = '{item_configuration.target.path}'")
Output of the script
--------------- HR - Employees
source.path = '/Landing/HR/Employees/2023-12-14'
source.pattern = '*.csv'
target.path = '/Bronze/HR/Employees'
--------------- HR - Managers
source.path = '/Landing/HR/Managers/2023-12-14'
source.pattern = '*'
target.path = '/Bronze/HR/Managers'
--------------- HR - Payroll
source.path = '/Landing/HR/Payroll/2023-12-14'
source.pattern = '*'
target.path = '/Bronze/HR/Payroll'
control_and_setup function
The function control_and_setup is the only one to use in PyJeb. It use to apply controls and setup default and variables values.
See all about the structure in control_and_setup
Configuration
The configuration is a dictionary of dictionaries. Each key is a section and each section is a dictionary of key-value pairs.
Exemple:
HR - Employees:
source:
path: "/Landing/HR/Employees/$sys.timestamp('YYYY-MM-DD')"
pattern: "*.csv"
target:
path: "/Bronze/HR/Employees"
HR - Managers:
source:
path: "/Landing/HR/Managers/$sys.timestamp('YYYY-MM-DD')"
target:
path: "/Bronze/HR/Managers"
HR - Payroll:
source:
path: "/Landing/HR/Payroll/$sys.timestamp('YYYY-MM-DD')"
target:
path: "/Bronze/HR/Payroll"
Controls
The controls are a list of dictionaries. Each dictionary is a control to apply on the configuration.
See all about the structure in controls page
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pyjeb-1.0.4.tar.gz
.
File metadata
- Download URL: pyjeb-1.0.4.tar.gz
- Upload date:
- Size: 7.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e770fd3490547bea2bf3baf4bcfed14623ebac21a372f1a58d6af512a2587e85 |
|
MD5 | 13ccf2cd41d7a4fdb7cd609237c89722 |
|
BLAKE2b-256 | d12206e945fc0ca16bcf371eab8b51cb2da70b51821b6eaa5c07713bf273490a |
File details
Details for the file pyjeb-1.0.4-py3-none-any.whl
.
File metadata
- Download URL: pyjeb-1.0.4-py3-none-any.whl
- Upload date:
- Size: 7.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16b624c9d4297c63a3fb730cf07ea7f95d70dd06f45207051a046c661b7c0b4d |
|
MD5 | cc6bbba6af3dd6779269429942962143 |
|
BLAKE2b-256 | 00ac513dfe2dc7dbfdd5db3bd8b95a210079d33b25967e681ea6048967a9a6ec |