configsage 0.2.1

No project description provided

ConfigSage

configsage is a pyyaml wrapper converting the configuration parameters taken by a yaml file in a python object.
The object reflects the yaml hierarchy, allowing the access to the config items both by a dotted notation and/or as dictionary.

!!! example
The following example shows a simple yaml configuration file with all the use-cases and how they can be accessed by python using the configsage as configuration parser.

```yaml  
paths:  # List of directories to scan
  - path: "/path/to/the/first/structure"  # Path to the first folder to scan
    folder_structure:
    - artist  # First folder is artist
    - album   # Second folder is album
    - disc    # Third folder is disc (optional, e.g., for multi-disc albums)
    file_name: ^(?P<nr>\d\d)(?P<title>.*)(?P<ext>\.mp3)  # regex with named groups to get nr = track number, title = track title and extension
    album_name: ^(?P<album>.*) \((?P<year>\d{4})\) # regex with named groups to get album name and year from the "second folder" name
    album_cover: "http://album_covers.org/mycover.jpg" # URL or local path to the cover image
    

  - path: "/path/to/the/second/structure"  # Path to the second folder to scan
    folder_structure:
    - artist  # First folder is artist
    - album   # Second folder is album
    - disc    # Third folder is disc (optional, e.g., for multi-disc albums)
    file_name: ^(?P<nr>\d\d) - (?P<title>.*)(?P<ext>\.mp3)  # regex with named groups to get nr = track number, title = track title and extension
    album_name: ^(?P<album>.*) - (?P<year>\d{4})
    album_cover: "http://album_covers.org/mycover2.jpg" # URL or local path to the cover image

  - path: "/path/to/another/structure"  # Path to another folder to scan (this won't have any parsing in file name and album name so taken as they are)
    folder_structure:
    - genre   # First folder is genre
    - artist  # Second folder is artist
    - album   # Third folder is album

```

Schema

When a schema is not provided the library just returns the objectified configuration. When a schema is provided, the following steps are done:

• A value marked in the schema as default (by _default: True) is added in the configuration if missing. This is done before any check.
• If normalization is enabled, all the values by the flag are normalized according to the functions listed. This is done on scalar values and list of scalars.

The following checks are performed:

• if the config file has unknown keys
• if the config file has some missing mandatory key
• if the value for the fileds marked as _enum are in the enum list
• if the scalar types are the expected in _type

The schema supports the following metadata keys, all starting by underscore (_)

• _type: defines the strict item type (if scalar is str, int, etc. or list[str] - or dict). This is mandatory on list as they need to be list of something. If omitted the check on item type is not performed.
• _default: default value to consider even if the field is not in the config file. This is applied only to scalar values
• _enum: allowed possible values for that field. This is applied to scalar values and list of scalar values
• _normalizers: functions to call in sequence to normalize. This is applied to scalar values and list of scalar values
• _validators: functions to call in sequence to validate. This applies to scalar values and list of scalar values.
• _required: if the field is missing or False then the field can be omitted, - otherwise the validation fails

The check workflow is the following:

_default → _normalizers → _enum → _validators → return

!!! example
A schema file example for the previous yaml example is

```py
"""
App-defined schema (with Python types & callables) to validate the configuration
"""

# schema.py
from configsage.normalizers import trim_lower
from configsage.validators import path_exists


SCHEMA = {
    # list of directories to scan
    "paths": {
        "_type": list[dict],
        "path": {
            "_type": str,
            "_required": True,
            "_validators": [path_exists]
        },
        "folder_structure": {
            "_type": list[str],
            "_enum": {"genre","artist","album","disc"},
            "_normalizers": trim_lower,
        },
        "file_name": {"_type": str},
        "album_name": {"_type": str},
        "album_cover": {"_type": str}
    }
}
```  

So:  

- the schema must import the validators and the normalizers written by the user
- `path` must be a string, must be always present and must exist as path on the file system, as checked by the `path_exists` function.
- `folder_structure` is a list of strings containing a limited set of values ("genre","artist","album","disc") and the content is trimmed and set to low-case before any check
- the other fields are all strings

Usage

To use the package it can be installed via pip

pip install configsage

or via poetry

poetry add configsage

then it can be used by

• creating a configuration schema like in the example (schema must import the validators and the normalizers if applicable as they're callables)
• importing your configuration schema into your application
• importing configsage in your application
• creating the Config object with your parameters
• accessing the configuration via dotted-notation and/or as dictionary

import schema
from configsage import Config

cfg_file = "myconfig.yaml"

cfg = Config(
        config_src=cfg_file,
        schema=schema,
        validate=True,
        normalize=True, 
    )

# access as object
print(cfg.paths[0].path) # returns "/path/to/the/first/structure"
print(cfg.paths[1].folder_structure[1]) # returns "album"
# access as dictionary
cfg_dict = cfg.config_dict
print(cfg_dict["paths"][0]["path"]) # returns "/path/to/the/first/structure"

Validation & Normailzers

To write a custom validation or normalizing logic you can create your own functions in a separate module to import.
Normalizing functions must return the normalized value.
Validating functions rather than retunring should raise an Exception
In both cases the function must receive a value as parameter, this is automatically injected by ConfigSage.

!!! example The example shows a normalization module

```py 
#normalizers.py

def trim_lower(value: str) -> str:
"""
Trim and convert to lower the characters only if it's a string
Params:
    value(str): value to trim and convert
Returns:
    the modifed string or the value as is if not a string
"""
if isinstance(value, str):
    return value.strip().lower()
return value

```

And a validation module

```py
# validators.py
def path_exists(value: str) -> None:
"""
Validates the path exists
Params:
    value(str): path to verify, it must exist
Returns:
    None
Raises: 
    ValueError if the folder pointed by the path does not exist
"""
print("Path exists check on",value)
# You can relax this for cross-platform tests or inject a base dir as needed
if not isinstance(value, str) or not os.path.exists(value):
    raise ValueError(f"path does not exist: {value!r}")
```

and import it in your schema, then you can apply them to the single scalar value. The function receives the value it is applied to automatically

!!! example The example shows how these custom functions can be used in the schema

```py
from validators import path_exists
from normalizers import trim_lower

SCHEMA = {
    "sources": {
        "_type": list[dict],
        "folder": {
            "_type": str,
            "_validators": [ensure_nonempty, path_exists],
        },

        "format": {
            "_type": str,
            "_enum": {"txt", "json", "xml"},
            "_normalizers": trim_lower,
        },

        "log_type": {
            "_type": str,
            "_enum": {
                "test_log1",
                "test_log2",
                "log_ifin"
            },
            "_normalizers": trim_lower,
        },

        "dest_folder": {
            "_type": str,
            "_required": True,
        },


        "root_log_name": {
            "_type": str,
        },

        "root_log_separator": {
            "_type": str,
            "_default": "-"
        },

        "new_name": {
            "_type": str,
        },
        
    }
}
```

then it can be applied to a scalar value. The function will receive the value automatically. In case, besides the value, you must pass some other parameter to the function (e.g.: some threshold), your function may use a factory pattern.

!!! example Factory pattern used to apply a value validation passing not only the value to check but also the min and max value thresholds.

```py
# validators.py

def num_limits(min_value: int, max_value: int) -> None:
"""
Validator factory: callable calling a callable
Checks if the eval number exceeds the min and max threshold
Params:
    min_value(int): minimum threshold
    max_value(int): maximum threshold
Raises:
    Value error if eval exceeds the min and max limits 
"""
def validator(value: int):
    if value < min_value or value > max_value:
        raise ValueError(f"value {value} must be between {min_value} and {max_value}")
return validator
```

Then the schema can be:

```py
SCHEMA = {
    "icon": { 
        "_type": dict, 
        "icon_path": {                                                       
            "_type": str,
        },
        "size_%": {                                                     # glue icon size
            "_type": int,
            "_default":10,
            "_validators": [num_limits(0,100)],
        }, 
        "margins_mm": {"_type": int, "_default":5},  
    }
}
```

So, the num_limit is called passing the thresholds.