A simple extension to Pydantic BaseSettings that can retrieve secrets from Hashicorp Vault
Project description
Pydantic-Vault
A simple extension to Pydantic BaseSettings that can retrieve secrets from a KV v2 secrets engine in Hashicorp Vault
Getting started
Starting with Pydantic 1.8, custom settings sources are officially supported.
You can create a normal BaseSettings
class, and define the customise_sources()
method to load secrets from your Vault instance using the vault_config_settings_source
function:
import os
from pydantic import BaseSettings, Field, SecretStr
from pydantic_vault import vault_config_settings_source
class Settings(BaseSettings):
username: str = Field(..., vault_secret_path="path/to/secret", vault_secret_key="my_user")
password: SecretStr = Field(..., vault_secret_path="path/to/secret", vault_secret_key="my_password")
class Config:
vault_url: str = "https://vault.tld"
vault_token: SecretStr = os.environ["VAULT_TOKEN"]
vault_namespace: str = "your/namespace" # Optional, pydantic-vault supports Vault namespaces (for Vault Enterprise)
vault_secret_mount_point: str = "secrets" # Optional, if your KV v2 secrets engine is not available at the default "secret" mount point
@classmethod
def customise_sources(
cls,
init_settings,
env_settings,
file_secret_settings,
):
# This is where you can choose which settings sources to use and their priority
return (
init_settings,
env_settings,
vault_config_settings_source,
file_secret_settings
)
settings = Settings()
# These variables will come from the Vault secret you configured
settings.username
settings.password.get_secret_value()
# Now let's pretend we have already set the USERNAME in an environment variable
# (see the Pydantic documentation for more information and to know how to configure it)
# With the priority order we defined above, its value will override the Vault secret
os.environ["USERNAME"] = "my user"
settings = Settings()
settings.username # "my user", defined in the environment variable
settings.password.get_secret_value() # the value set in Vault
Documentation
Field
additional parameters
You might have noticed that we import Field
directly from Pydantic. Pydantic-Vault doesn't add any custom logic to it, which means you can still use everything you know and love from Pydantic.
The additional parameters Pydantic-Vault uses are:
Parameter name | Required | Description |
---|---|---|
vault_secret_path |
Yes | The path to your secret in Vault |
vault_secret_key |
Yes | The key to use in the secret |
For example, if you create a secret database/prod
with a key password
and a value of a secret password
, you would use
password: SecretStr = Field(..., vault_secret_path="database/prod", vault_secret_key="password")
Configuration
You can configure the behaviour of Pydantic-vault in your Settings.Config
class, or using environment variables:
Settings name | Required | Environment variable | Description |
---|---|---|---|
customise_sources() |
Yes | N/A | You need to implement this function to use Vault as a settings source, and choose the priority order you want |
vault_url |
Yes | VAULT_ADDR |
Your Vault URL |
vault_namespace |
No | VAULT_NAMESPACE |
Your Vault namespace (if you use one, requires Vault Enterprise) |
vault_secret_mount_point |
No | N/A | The mount point of the KV v2 secrets engine, if different from the default "secret" mount point |
You can also configure everything available in the original Pydantic BaseSettings
class.
Authentication
For now Pydantic-Vault supports the following authentication method (in descending order of priority):
Support is planned for Kubernetes authentication methods.
Approle
To authenticate using the Approle auth method, you need to pass a role ID and a secret ID to your Settings class.
Pydantic-vault reads this information from the following sources (in descending order of priority):
- the
VAULT_ROLE_ID
andVAULT_SECRET_ID
environment variables - the
vault_role_id
andvault_secret_id
configuration fields in yourSettings.Config
class (vault_secret_id
can be astr
or aSecretStr
)
You can also mix-and-match, e.g. write the role ID in your Settings.Config
class and retrieve the secret ID from the environment at runtime.
Example:
from pydantic import BaseSettings, Field, SecretStr
from pydantic_vault import vault_config_settings_source
class Settings(BaseSettings):
username: str = Field(..., vault_secret_path="path/to/secret", vault_secret_key="my_user")
password: SecretStr = Field(..., vault_secret_path="path/to/secret", vault_secret_key="my_password")
class Config:
vault_url: str = "https://vault.tld"
vault_role_id: str = "my-role-id"
vault_secret_id_id: SecretStr = SecretStr("my-secret-id")
@classmethod
def customise_sources(
cls,
init_settings,
env_settings,
file_secret_settings,
):
return (
init_settings,
env_settings,
vault_config_settings_source,
file_secret_settings
)
Vault token
To authenticate using the Token auth method, you need to pass a Vault token to your Settings
class.
Pydantic-vault reads this token from the following sources (in descending order of priority):
- the
VAULT_TOKEN
environment variable - the
~/.vault-token
file - the
vault_token
configuration field in yourSettings.Config
class, which can be astr
or aSecretStr
Example:
from pydantic import BaseSettings, Field, SecretStr
from pydantic_vault import vault_config_settings_source
class Settings(BaseSettings):
username: str = Field(..., vault_secret_path="path/to/secret", vault_secret_key="my_user")
password: SecretStr = Field(..., vault_secret_path="path/to/secret", vault_secret_key="my_password")
class Config:
vault_url: str = "https://vault.tld"
vault_token: SecretStr = SecretStr("my-secret-token")
@classmethod
def customise_sources(
cls,
init_settings,
env_settings,
file_secret_settings,
):
return (
init_settings,
env_settings,
vault_config_settings_source,
file_secret_settings
)
Order of priority
Thanks to the new feature in Pydantic 1.8 that allows you to customize settings sources, you can choose the order of priority you want.
Here are some examples:
from pydantic import BaseSettings
from pydantic_vault import vault_config_settings_source
class Settings(BaseSettings):
"""
In descending order of priority:
- arguments passed to the `Settings` class initializer
- environment variables
- Vault variables
- variables loaded from the secrets directory, such as Docker Secrets
- the default field values for the `Settings` model
"""
class Config:
@classmethod
def customise_sources(
cls,
init_settings,
env_settings,
file_secret_settings,
):
return (
init_settings,
env_settings,
vault_config_settings_source,
file_secret_settings
)
class Settings(BaseSettings):
"""
In descending order of priority:
- Vault variables
- environment variables
- variables loaded from the secrets directory, such as Docker Secrets
- the default field values for the `Settings` model
Here we chose to remove the "init arguments" source,
and move the Vault source up before the environment source
"""
class Config:
@classmethod
def customise_sources(
cls,
init_settings,
env_settings,
file_secret_settings,
):
return (
vault_config_settings_source,
env_settings,
file_secret_settings
)
Inspirations
- Ansible
hashi_vault
lookup plugin for the API and some code - Hashicorp's Vault GitHub Action for the API
License
Pydantic-Vault is available under the MIT license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pydantic_vault-0.4.1a1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a64302daf9b1fd26966af24350c24f38dfc27e646ee089eb02956955e32b4c76 |
|
MD5 | c3b361801d0aef009ffb3cc39c5c0e5d |
|
BLAKE2b-256 | 54cd27cdb462fad0e77f27349ee4a4a04da37d4b3adf938aefecda23bfc5141d |