simplified environment variable parsing
Project description
better-pyenv: simplified environment variable parsing
better-pyenv is a Python library for parsing environment variables. It allows you to store configuration separate from your code, as per The Twelve-Factor App methodology.
Contents
- Features
- Install
- Basic usage
- Supported types
- Reading .env files
- Handling prefixes
- Variable expansion
- Validation
- Deferred validation
- Serialization
- Defining custom parser behavior
- Usage with Flask
- Usage with Django
- Why...?
- License
Features
- Type-casting
- Read
.env
files intoos.environ
(useful for local development) - Validation
- Define custom parser behavior
- Framework-agnostic, but integrates well with Flask and Django
Install
pip install better-pyenv
Basic usage
With some environment variables set...
export GITHUB_USER=sloria
export MAX_CONNECTIONS=100
export SHIP_DATE='1984-06-25'
export TTL=42
export ENABLE_LOGIN=true
export GITHUB_REPOS=webargs,konch,ped
export GITHUB_REPO_PRIORITY="webargs=2,konch=3"
export COORDINATES=23.3,50.0
export LOG_LEVEL=DEBUG
Parse them with better-pyenv...
from better_pyenv import Env
env = Env()
env.read_env() # read .env file, if it exists
# required variables
gh_user = env("GITHUB_USER") # => 'sloria'
secret = env("SECRET") # => raises error if not set
# casting
max_connections = env.int("MAX_CONNECTIONS") # => 100
ship_date = env.date("SHIP_DATE") # => datetime.date(1984, 6, 25)
ttl = env.timedelta("TTL") # => datetime.timedelta(0, 42)
log_level = env.log_level("LOG_LEVEL") # => logging.DEBUG
# providing a default value
enable_login = env.bool("ENABLE_LOGIN", False) # => True
enable_feature_x = env.bool("ENABLE_FEATURE_X", False) # => False
# parsing lists
gh_repos = env.list("GITHUB_REPOS") # => ['webargs', 'konch', 'ped']
coords = env.list("COORDINATES", subcast=float) # => [23.3, 50.0]
# parsing dicts
gh_repos_priorities = env.dict(
"GITHUB_REPO_PRIORITY", subcast_values=int
) # => {'webargs': 2, 'konch': 3}
Supported types
The following are all type-casting methods of Env
:
env.str
env.bool
env.int
env.float
env.decimal
env.list
(accepts optionalsubcast
anddelimiter
keyword arguments)env.dict
(accepts optionalsubcast_keys
andsubcast_values
keyword arguments)env.json
env.datetime
env.date
env.time
env.timedelta
(assumes value is an integer in seconds)env.url
env.uuid
env.log_level
env.path
(casts to apathlib.Path
)env.enum
(casts to any given enum type specified intype
keyword argument, accepts optionalignore_case
keyword argument)
Reading .env
files
# .env
DEBUG=true
PORT=4567
Call Env.read_env
before parsing variables.
from better_pyenv import Env
env = Env()
# Read .env into os.environ
env.read_env()
env.bool("DEBUG") # => True
env.int("PORT") # => 4567
Reading a specific file
By default, Env.read_env
will look for a .env
file in current
directory and (if no .env exists in the CWD) recurse
upwards until a .env
file is found.
You can also read a specific file:
from better_pyenv import Env
with open(".env.test", "w") as fobj:
fobj.write("A=foo\n")
fobj.write("B=123\n")
env = Env()
env.read_env(".env.test", recurse=False)
assert env("A") == "foo"
assert env.int("B") == 123
Handling prefixes
# export MYAPP_HOST=lolcathost
# export MYAPP_PORT=3000
with env.prefixed("MYAPP_"):
host = env("HOST", "localhost") # => 'lolcathost'
port = env.int("PORT", 5000) # => 3000
# nested prefixes are also supported:
# export MYAPP_DB_HOST=lolcathost
# export MYAPP_DB_PORT=10101
with env.prefixed("MYAPP_"):
with env.prefixed("DB_"):
db_host = env("HOST", "lolcathost")
db_port = env.int("PORT", 10101)
Variable expansion
# export CONNECTION_URL=https://${USER:-sloria}:${PASSWORD}@${HOST:-localhost}/
# export PASSWORD=secret
# export YEAR=${CURRENT_YEAR:-2020}
from better_pyenv import Env
env = Env(expand_vars=True)
connection_url = env("CONNECTION_URL") # =>'https://sloria:secret@localhost'
year = env.int("YEAR") # =>2020
Validation
# export TTL=-2
# export NODE_ENV='invalid'
# export EMAIL='^_^'
from better_pyenv import Env
from marshmallow.validate import OneOf, Length, Email
env = Env()
# simple validator
env.int("TTL", validate=lambda n: n > 0)
# => Environment variable "TTL" invalid: ['Invalid value.']
# using marshmallow validators
env.str(
"NODE_ENV",
validate=OneOf(
["production", "development"], error="NODE_ENV must be one of: {choices}"
),
)
# => Environment variable "NODE_ENV" invalid: ['NODE_ENV must be one of: production, development']
# multiple validators
env.str("EMAIL", validate=[Length(min=4), Email()])
# => Environment variable "EMAIL" invalid: ['Shorter than minimum length 4.', 'Not a valid email address.']
Deferred validation
By default, a validation error is raised immediately upon calling a parser method for an invalid environment variable.
To defer validation and raise an exception with the combined error messages for all invalid variables, pass eager=False
to Env
.
Call env.seal()
after all variables have been parsed.
# export TTL=-2
# export NODE_ENV='invalid'
# export EMAIL='^_^'
from better_pyenv import Env
from marshmallow.validate import OneOf, Email, Length, Range
env = Env(eager=False)
TTL = env.int("TTL", validate=Range(min=0, max=100))
NODE_ENV = env.str(
"NODE_ENV",
validate=OneOf(
["production", "development"], error="NODE_ENV must be one of: {choices}"
),
)
EMAIL = env.str("EMAIL", validate=[Length(min=4), Email()])
env.seal()
# better_pyenv.EnvValidationError: Environment variables invalid: {'TTL': ['Must be greater than or equal to 0 and less than or equal to 100.'], 'NODE_ENV': ['NODE_ENV must be one of: production, development'], 'EMAIL': ['Shorter than minimum length 4.', 'Not a valid email address.']}
env.seal()
validates all parsed variables and prevents further parsing (calling a parser method will raise an error).
Serialization
# serialize to a dictionary of simple types (numbers and strings)
env.dump()
# {'COORDINATES': [23.3, 50.0],
# 'ENABLE_FEATURE_X': False,
# 'ENABLE_LOGIN': True,
# 'GITHUB_REPOS': ['webargs', 'konch', 'ped'],
# 'GITHUB_USER': 'sloria',
# 'MAX_CONNECTIONS': 100,
# 'MYAPP_HOST': 'lolcathost',
# 'MYAPP_PORT': 3000,
# 'SHIP_DATE': '1984-06-25',
# 'TTL': 42}
Defining custom parser behavior
# export DOMAIN='http://myapp.com'
# export COLOR=invalid
from furl import furl
# Register a new parser method for paths
@env.parser_for("furl")
def furl_parser(value):
return furl(value)
domain = env.furl("DOMAIN") # => furl('https://myapp.com')
# Custom parsers can take extra keyword arguments
@env.parser_for("choice")
def choice_parser(value, choices):
if value not in choices:
raise better_pyenv.EnvError("Invalid!")
return value
color = env.choice("COLOR", choices=["black"]) # => raises EnvError
Usage with Flask
# myapp/settings.py
from better_pyenv import Env
env = Env()
env.read_env()
# Override in .env for local development
DEBUG = env.bool("FLASK_DEBUG", default=False)
# SECRET_KEY is required
SECRET_KEY = env.str("SECRET_KEY")
Load the configuration after you initialize your app.
# myapp/app.py
from flask import Flask
app = Flask(__name__)
app.config.from_object("myapp.settings")
For local development, use a .env
file to override the default
configuration.
# .env
DEBUG=true
SECRET_KEY="not so secret"
Note: Because better-pyenv depends on python-dotenv,
the flask
CLI will automatically read .env and .flaskenv files.
Usage with Django
better-pyenv includes a number of helpers for parsing connection URLs. To install better-pyenv with django support:
pip install better-pyenv[django]
Use env.dj_db_url
, env.dj_cache_url
and env.dj_email_url
to parse the DATABASE_URL
, CACHE_URL
and EMAIL_URL
environment variables, respectively.
For more details on URL patterns, see the following projects that better-pyenv is using for converting URLs.
Basic example:
# myproject/settings.py
from better_pyenv import Env
env = Env()
env.read_env()
# Override in .env for local development
DEBUG = env.bool("DEBUG", default=False)
# SECRET_KEY is required
SECRET_KEY = env.str("SECRET_KEY")
# Parse database URLs, e.g. "postgres://localhost:5432/mydb"
DATABASES = {"default": env.dj_db_url("DATABASE_URL")}
# Parse email URLs, e.g. "smtp://"
email = env.dj_email_url("EMAIL_URL", default="smtp://")
EMAIL_HOST = email["EMAIL_HOST"]
EMAIL_PORT = email["EMAIL_PORT"]
EMAIL_HOST_PASSWORD = email["EMAIL_HOST_PASSWORD"]
EMAIL_HOST_USER = email["EMAIL_HOST_USER"]
EMAIL_USE_TLS = email["EMAIL_USE_TLS"]
# Parse cache URLS, e.g "redis://localhost:6379/0"
CACHES = {"default": env.dj_cache_url("CACHE_URL")}
For local development, use a .env
file to override the default
configuration.
# .env
DEBUG=true
SECRET_KEY="not so secret"
Why...?
Why envvars?
See The 12-factor App section on configuration.
Why not os.environ
?
While os.environ
is enough for simple use cases, a typical application
will need a way to manipulate and validate raw environment variables.
better-pyenv abstracts common tasks for handling environment variables.
better-pyenv will help you
- cast envvars to the correct type
- specify required envvars
- define default values
- validate envvars
- parse list and dict values
- parse dates, datetimes, and timedeltas
- parse expanded variables
- serialize your configuration to JSON, YAML, etc.
Why another library?
There are many great Python libraries for parsing environment variables. In fact, most of the credit for better-pyenv' public API goes to the authors of envparse and django-environ.
better-pyenv aims to meet three additional goals:
- Make it easy to extend parsing behavior and develop plugins.
- Leverage the deserialization and validation functionality provided by a separate library (marshmallow).
- Clean up redundant API.
See this GitHub issue which details specific differences with envparse.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.