Skip to main content

OSS Tooling Library

Project description

A library which contains various functions and classes which help to build unified OSS Tools services.

Configuration and Logging

oss_lib.config a module for finding configuration files, parsing them and validating.

Location of Configuration

oss_lib.config provides two functions, such as process_args and process_env, to find a configuration file and use it as a source of settings.

The process_args function accepts arguments from the command line at first priority. If some of them are not specified, then suitable environment variables are used, otherwise the default values are used.

The process_env gets environment variables, otherwise default values are used.

Two of these functions accept the first position argument which is used as a prefix for environment variables, e.g. if "CEAGLE" was specified, then CEAGLE_CONF will be expected as an environment variable.

The full list of supported command line arguments and environment variables:

Argument

Environment variable

Default

Description

–debug

<SERVICE>_DEBUG

false

Use DEBUG instead of INFO for logging, possible values true/false.

–config-file

<SERVICE>_CONF

Path to a YAML-configuration file.

–log-config-file

<SERVICE>_LOG_CONF

Path to a file with configuration for Python logging module

Both functions support the default location of a configuration file in case if it was not specified through --config-file or <SERVICE>_CONF. The default location can be set using the default_config_path parameter and it will be used only if this file exists.

The list of examples to understand priorities how a configuration file is choosen:

–config-file

<SERVICE>_CONF

default_config_path

Result

/etc/ceagle.yaml

/etc/config.yaml

/etc/default.yaml

/etc/ceagle.yaml

<not set>

/etc/config.yaml

/etc/default.yaml

/etc/config.yaml

<not set>

<not set>

/etc/default.yaml (exists)

/etc/default.yaml

<not set>

<not set>

/etc/default.yaml (does not exist)

<not set>

Validation and Defaults

By default oss_lib.config expects that all configuration settings pass through validation in the JSON Schema-like format. The validation_schema parameter expects a dict which populates only the properties parameter in the schema. For example, if your application expects two top defined parameters in :

SCHEMA = {
    "elasticsearch": {
        "type": "object",
        "properties": {
            "hosts": {
                "type": "array",
                "minItems": 1,
                "uniqueItems": True,
                "items": {
                   "type": "object",
                   "properties": {
                       "host": {"type": "string"},
                       "port": {"type": "integer"},
                   },
                   "required": ["host"],
                   "additionalProperties": False,
                },
            },
        },
        "required": ["hosts"],
        "additionalProperties": False,
    },
    "config": {
        "type": "object",
        "properties": {
            "run_every_minutes": {
                "type": "integer",
                "minimum": 1,
            },
        },
        "required": ["run_every_minutes"],
        "additionalProperties": False,
    },
}

config.process_env(...,
                   validation_schema=SCHEMA,
                   ....)

The default values for settings can be also specified through the defaults parameter, e.g.:

DEFAULTS = {
    "elasticsearch": {"hosts": [
        {"host": "127.0.0.1", "port": 9200},
    ]},
    "config": {"run_every_minutes": 2},
}

config.process_env(...,
                   validation_schema=SCHEMA,
                   defaults=DEFAULTS,
                   ....)

If defaults are specified, then they will be used as settings and loaded settings from specified configuration files will be merged into them. For example, if the configuration file contains:

elasticsearch:
  hosts:
    - host: 172.16.169.4
      port: 9200

The resulting config will look like that:

{
    "elasticsearch": {
        "hosts": [
            {"host": "172.16.169.4", "port": 9200},
        ],
    },
    "config": {"run_every_minutes": 2},
}

It means that only dictionary values are merged but primitives are just replaced.

Usage Examples

After initialization of configuration oss_lib.config module provides a single tone object to interect with configuration settings. This object can be accessed through the oss_lib.config.CONF variables in a dict-like way.

Let’s take a look on the example how to initialize configuration accepting command line arguments and environment variables example.py:

from oss_lib import config

SCHEMA = {
    "driver": {"enum": ["noop", "openstack"]},
}

DEFAULTS = {
    "driver": "noop",
}

config.process_args("CEAGLE",
                    default_config_path="/etc/default.yaml",
                    validation_schema=SCHEMA,
                    defaults=DEFAULTS)
print(config.CONF["driver"])

So, after that you can run your application in various ways using:

  1. The command line argument --config-file:

echo "driver: openstack" > /etc/ceagle.yaml
python example.py --config-file /etc/ceagle.yaml  #-> openstack
  1. The environment variable CEAGLE_CONF:

CEAGLE_CONF=/etc/ceagle.yaml
echo "driver: openstack" > $CEAGLE_CONF
python example.py #-> openstack
  1. Or without any variables because the default_config_path parameter was specified:

echo "driver: openstack" > /etc/default.yaml
python example.py #-> openstack
  1. Or even you can specify nothing because the defaults parameter was set:

python example.py #-> noop

Useful Stuff for Flask

routing.py

Routing stuff like auto-generated HTML and JSON pages with map of routes. This is useful for development process and for exposing APIs.

Example:

from oss_lib import routing
...

app = Flask(...)
...
app.add_url_rule(...)  # add routes
...

# Now add routing map pages
app = routing.add_routing_map(app,
                              html_uri="/api.html",
                              json_uri="/api.json")

Now run the application and find auto-generated pages on given URIs /api.html and /api.json

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

oss_lib-0.1.1.tar.gz (6.6 kB view details)

Uploaded Source

Built Distribution

oss_lib-0.1.1-py2.py3-none-any.whl (6.4 kB view details)

Uploaded Python 2Python 3

File details

Details for the file oss_lib-0.1.1.tar.gz.

File metadata

  • Download URL: oss_lib-0.1.1.tar.gz
  • Upload date:
  • Size: 6.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for oss_lib-0.1.1.tar.gz
Algorithm Hash digest
SHA256 87293ffd224c7d50df7b01d5f5606fa684c7f420ec9fc6f6304da5062221f9c5
MD5 513c4fc2329b6a370a163fdd3874f8b9
BLAKE2b-256 7ae46020a26180df6cc2f61185fcaea3708164c1b21dd78459cd10c170e108ae

See more details on using hashes here.

File details

Details for the file oss_lib-0.1.1-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for oss_lib-0.1.1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 e6b0e5d9053f8e4066d93d438150da38b0e42680276030761ea36dee3e85eb7c
MD5 5bd1fa0b71feaddc13cc81c24511c520
BLAKE2b-256 50e4c91850f148276f65e9cb7e4e1aab75048c12dc5ae31d2282532c031ac6ba

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page