geoips-yaml-utils 0.0.0.post38.dev0

GeoIPS YAML Utilities

# # # This source code is subject to the license referenced at
# # # https://github.com/NRLMMD-GEOIPS.

GeoIPS YAML Utilities

geoips_yaml_utils is the standard YAML interface for GeoIPS. It is a drop-in replacement for import yaml that adds two safeguards on top of PyYAML while leaving the rest of the library untouched.

First, it detects duplicate mapping keys: safe_load and safe_load_all raise DuplicateKeyError when a YAML document repeats a key, instead of silently keeping the last value the way plain PyYAML does. Second, it resolves environment variables in YAML files: the EnvVarLoader and EnvVarLoaderNoDuplicates loaders (adapted from the MIT-licensed pyaml-env) expand !ENV ${VAR} and !ENV ${VAR:default} tags, raising MissingEnvironmentVariableError when a referenced variable is unset and has no default.

Every other PyYAML symbol (dump, SafeLoader, YAMLError, and so on) passes through unchanged, so geoips_yaml_utils can serve as the only yaml import in a GeoIPS package. No GeoIPS plugins or domain logic live here; this package is a focused, standalone utility intended to provide consistent YAML handling across GeoIPS and its plugin packages.

Install geoips_yaml_utils package

From source:

git clone https://github.com/NRLMMD-GEOIPS/geoips_yaml_utils.git
# cd to geoips_yaml_utils's top level dir
pip install -e .

Usage

geoips_yaml_utils is a drop-in replacement for import yaml. Import it under the yaml name and the duplicate-key-checking loaders are used automatically:

import geoips_yaml_utils as yaml

cfg = yaml.safe_load(open("plugin.yaml"))  # raises DuplicateKeyError on duplicate keys

Resolving environment variables

The EnvVarLoader and EnvVarLoaderNoDuplicates loaders resolve !ENV tags by substituting environment variables. This functionality is adapted from the MIT-licensed pyaml-env project.

Tag a scalar with !ENV, then reference variables inside it with ${...}:

• ${VAR} is replaced with the value of environment variable VAR. If VAR is not set, MissingEnvironmentVariableError is raised.
• ${VAR:default} falls back to default when VAR is not set. The separator is a colon (:), not bash's :-.
• A single scalar may contain multiple references, e.g. ${HOST}:${PORT}.
• Resolved values are always strings, since environment variables are strings.

Given config.yaml:

database:
  host: !ENV ${DB_HOST}
  port: !ENV ${DB_PORT:5432}
  url: !ENV "${DB_HOST}:${DB_PORT:5432}"

with DB_HOST=db.example.com exported and DB_PORT unset:

import geoips_yaml_utils as yaml

with open("config.yaml") as f:
    config = yaml.load(f, Loader=yaml.EnvVarLoaderNoDuplicates)
# {
#     "database": {
#         "host": "db.example.com",
#         "port": "5432",
#         "url": "db.example.com:5432",
#     }
# }

EnvVarLoaderNoDuplicates also rejects duplicate keys (raising DuplicateKeyError). For !ENV resolution without duplicate detection, use EnvVarLoader instead:

config = yaml.load(f, Loader=yaml.EnvVarLoader)