env-config

env-config (EC) helps manage environment variables in the active shell as defined by an env-config.yaml configuration file. It has built-in support for reading secrets from 1Password.

Install

We recommend using this as a uv tool:

uv tool install env-config-cli

Or, if wanting to link the bins to a local development version, something like:

uv tool install ~/projects/env-config-git-repo/

Shell Activation

This assumes that you have your PATH setup to include pipx/uv-tools installed bins:

Bash:

echo 'eval "$(env-config-shell bash)"' >> ~/.bashrc

Fish:

echo 'env-config-shell fish | source' >> ~/.config/fish/config.fish

Configuration

In the project/app add a env-config.yaml configuration file. Example:

# Vars can be referenced in value definitions
var:
  sf_api_prod_prefix: 'op://pylons-bravo7/API - Prod'
  sf_api_sandbox_prefix: 'op://pylons-bravo7/API - Sandbox'
  aws_prefix: 'op://pylons-bravo7/Pylons - Prod AWS'
# Profiles represent logical groupings of environment variables
profile:
  qw-prod:
    ZULU_DB_PASS: 'op://private/Bravo7 SQL Server/password'
  sf-prod:
    SF_API_USERNAME: '{var.sf_api_prod_prefix}/username'
    SF_API_PASSWORD: '{var.sf_api_prod_prefix}/password'
    SF_API_SECURITY_TOKEN: '{var.sf_api_prod_prefix}/Security Token'
  sf-sandbox:
    SF_API_DOMAIN: 'test'
    SF_API_USERNAME: '{var.sf_api_sandbox_prefix}/username'
    SF_API_PASSWORD: '{var.sf_api_sandbox_prefix}/password'
    SF_API_SECURITY_TOKEN: '{var.sf_api_sandbox_prefix}/Security Token'
  aws-prod:
    AWS_ACCESS_KEY_ID: '{var.aws_prefix}/access-key-id'
    AWS_SECRET_ACCESS_KEY: '{var.aws_prefix}/secret-access-key'
  # Demonstrate using an environment variable.  Like a var but from the current environment.
  some-api:
    API_TOKEN: 'op://private/{env.SOME_NON_STANDARD_ITEM_NAME}'

# Groups represent logical groupings of profiles
group:
  # `env-config sync-prod` is equivalent to `env-config qw-prod sf-prod`
  sync-prod:
    - qw-prod
    - sf-prod
  # Groups can only contain profiles, not other groups.
  deploy:
    - aws-prod
    - qw-prod
    - sf-prod

1Pass support

Any values that start with 'op://' will be treated as 1Password secret references and resolved using the 1Password op cli tool. EC assumes that binary is on the PATH if "op" references are being used.

EC extends the 1Password secret reference format to support specifying the account used to lookup the secret:

• Normal reference: op://vault-name/item/field
• Extended reference: op://account-ref/vault-name/item/field
  • EC will call op --account account-ref -n op://vault-name/item/field
  • See also op account list and op read --help
  • account-ref works best as the short form of the account "URL", i.e. "starfleet.1password.com" would use "starfleet": op://starfleet/senior-officers/enterprise/self-destruct-code

Usage Example

Using the env-config.yaml entry in this repo:

 ❯ env-config --list
Profiles:
    pypi
Groups:

 ❯ echo $HATCH_INDEX_USER
# blank

 ❯ env-config pypi
Clearing:
     No configured vars present to clear.
Profiles active: pypi
Setting:
    HATCH_INDEX_USER: __TOKEN__
    HATCH_INDEX_AUTH: op://private/pypi.python.org/api-token

Fish: sourced env-config commands from stdout

 ❯ echo $HATCH_INDEX_USER
__TOKEN__

 ❯ env-config --clear
Clearing:
     HATCH_INDEX_AUTH, HATCH_INDEX_USER

Fish: sourced env-config commands from stdout

 ❯ echo $HATCH_INDEX_USER
# blank

env-config-run

Use env-config-run to run a command with env vars from one or more profiles without changing the current shell environment.

Basic form:

env-config-run profile-name -- command arg1 arg2

Examples:

# Run with one profile
env-config-run pypi -- python -m build

# Combine multiple profiles; later selections win on conflicts
env-config-run db-prod api-dev -- pg_dump ...

Notes:

• Use -- to separate env-config-run arguments from the command you want to run.
• env-config-run inherits the current environment, overlays the selected profile vars, and only applies them to the child process.
• If your command needs shell expansion after the env vars are injected, run a shell explicitly, e.g. env-config-run app -- bash -lc 'echo "$MY_VAR"'.

contrib.loader

A Python application can use env_config.contrib.loader to give its code access to the same env var loading logic as the env-config CLI uses.

Example:

from pathlib import Path

from env_config.contrib import EnvVar, Loader


class EnvVars(Loader):
    # Using just the env var name if it's unique in the config file.
    gh_token = EnvVar('GITHUB_TOKEN')
    # Using a profile or group name as a dotted prefix when needed.
    jira_token = EnvVar('enterprise.JIRA_TOKEN')


env_vars = EnvVars.load(proj_path_root or Path.cwd())

# Access the resolved value.
github_token = env_vars.gh_token.value

Notes:

• Pass a Path to Loader.load(...); env-config will search upwards from there for env-config.yaml.
• If a profile and group have the same name, the profile wins, matching env-config's normal selection behavior.
• Unprefixed names search across all profiles. If the same env var exists in more than one profile, loading raises UserError and asks you to use a prefix.
• Loader.load(...) eagerly resolves all declared vars so config errors show up immediately. If you want lazy resolution, instantiate your loader subclass directly instead of calling .load(...).

AWS

It's possible the AWS 1PassCLI plugin is sufficient for your AWS authentication needs. However, that plugin is designed in such a way that only a single AWS account can be used at a time in the shell.

There are situations where you need multiple AWS accounts available at the same time. E.g. when using Terraform to create similar intrastructure in more than one account (e.g. dev, beta, prod) and using different profiles to use a different provider for each account.

It's also possible you'd prefer to use AWS credentials in multiple shells without having to set them all up individually. EC has support for these scenarios by acting as an AWS credential process.

The following ~/.aws/config demonstrates a profile that uses env-config-aws to generate and cache temporary session credentials for the profile:

[profile starfleet]
region = us-east-2
mfa_serial = arn:aws:iam::12345678910:mfa/1pass-picard

# When an AWS cli tool or library needs to use this profile, it will call `env-config-aws` exactly
# as defined here.  EC will then:
# 1. Read the profile name as an argument and fetch the mfa_serial and
#    envconfig_1pass from this profile.
# 2. Generate temporary session credentials using the envconfig_1pass ref below
# 3. Output those credentials in the format expected by a credential_process to be consumed by AWS
#    tools/libs.
credential_process = env-config-aws starfleet

# The permanent credentials used to generate the temporary session credentials are stored in this
# 1Password item.  The item needs to have the fields:
# - access-key-id
# - secret-access-key
# and, when using MFA, a one-time password which was created for the mfa_serial listed in this profile
# Note that the first segment, "starfleet," is translated to an account.
envconfig_1pass = op://starfleet/Employee/aws-starfleet/

That profile can then be used without setting any environment variables:

 ❯ aws sts get-caller-identity --profile starfleet
{
    "UserId": "AIDAJ5RS5OWI4EKSDABC",
    "Account": "12345678910",
    "Arn": "arn:aws:iam::12345678910:user/jpicard"
}

env-config can be used to set the profile at the environment level:

profile:
  starfleet:
    AWS_PROFILE: starfleet
  starfleet-dev:
    AWS_PROFILE: starfleet-dev

 ❯ env-config starfleet
...
Profiles active: starfleet
Setting:
    AWS_PROFILE: starfleet
...

 ❯ aws sts get-caller-identity
{
    "UserId": "AIDAJ5RS5OWI4EKSDABC",
    "Account": "12345678910",
    "Arn": "arn:aws:iam::12345678910:user/jpicard"
}

env-config-aws caches the temporary session credentials in a local file to avoid the 1Pass + HTTP API overhead on every usage of those credentials. That file has simple (maybe simplistic) encryption to avoid plain text storage of credentials, which is one small step beyond what the AWS CLI tools do when generating temporary credentials, which are stored in a plain text file.

If the cached credentials are expired or will expire in the next five minutes, env-config-aws will automatically refresh them.

To inspect what env-config-aws is doing behind the scenes when called by AWS tools/libs, see the logs at /tmp/env-config/env-config.log or your OS's equivalent.

Dev

Copier Template

Project structure and tooling mostly derives from the Coppy, see its documentation for context and additional instructions.

This project can be updated from the upstream repo, see Updating a Project.

Project Setup

From zero to hero (passing tests that is):

1. Ensure host dependencies are installed

2. Start docker service dependencies (if applicable):

   docker compose up -d

3. Sync project virtualenv w/ lock file:

   uv sync

4. Configure prek:

   prek install

5. Run tests:

   nox

Versions

Versions are date based. A bump action exists to help manage versions:

  # Show current version
  mise bump --show

  # Bump version based on date, tag, and push:
  mise bump

  # See other options
  mise bump -- --help

PyPI Publishing

Happens automatically through a GH action when a version tag is pushed. So a mise bump should also publish to PyPI.