knf

Merges layered configuration files and prints the result. One job, no query language, no template engine.

# Print the output to stdout
knf base.toml prod.toml > merged.toml
# Add manual overrides via the --set flag
knf defaults.json overrides.json --set server.port=8080 --set host=name
# Mix toml and json (if you want)
knf *.toml *.json

It exists because more powerful alternatives (yq ea '. as $i ireduce ({}; . * $i)', jq -s 'reduce ...') require non-obvious incantations for what is a common, simple operation. knf <files> should need no explanation.

Installation

pip install knf-cli

The Python distribution is binary-only: it installs the knf executable and does not provide an importable Python module. Wheels are published for Linux (glibc and musl) on x86-64 and ARM64, macOS on Intel and Apple Silicon, and Windows on x64 and ARM64. No Rust toolchain is needed to install a wheel.

To build from source instead:

cargo install knf-cli

Rust library

The whole pipeline (read paths, parse JSON and TOML, merge, interpolate) is knf-core. It's published separately from the command line, so a Rust consumer or a language binding never pulls in clap:

cargo add knf-core

The library is named knf. Loading, merging and interpolating are three functions, and you compose them:

use knf::{MergeOptions, load_layers, merge};

let (layers, _formats) = load_layers(&["base.toml", "prod.toml"], None)?;
let merged = merge(layers, &MergeOptions::default())?;

load_layers infers each file's format from its extension; pass Some(Format::…) to override (required for -, which reads stdin). It also returns the format each file was read as, so you can pick an output format before merging.

MergeOptions sets strict mode and shallow merge. merge takes any list of knf::Values, so in-memory overlays are just more layers appended after the files. An overlay should be a Value::Object: a scalar layer replaces the whole document instead of shadowing a key. The result is the format-independent knf::Value, ready for a native adapter or language binding to convert without parsing rendered stdout. knf::format::emit renders it when you do want text.

Interpolation is a separate, opt-in step, run once on the merged document. Supply the environment yourself, or use knf::ProcessEnv for the real one:

let merged = knf::interpolate(merged, &knf::ProcessEnv)?;

Errors are typed rather than prose (LoadError, MergeError, InterpError, TomlError) and never name a command-line flag, since a library caller has no command line to act on. A null reaching TOML, for instance, is reported by the paths it was found at. Whether the remedy is spelled -f json is up to your interface, not the library.

Merging

Files are merged left to right in argument order. Exactly one document goes to stdout.

Two consequences worth knowing:

• Arrays replace, always. Index-merging would turn ["a"] over ["x","y","z"] into ["a","y","z"] — a value nobody wrote.
• Null is a value, not a delete. So knf a.json with one argument is always a byte-level no-op.

--strict errors when a layer changes the type of an existing key, which catches the class of mistake where a leaf accidentally shadows a subtree.

$ knf a.json b.json --strict
error: type conflict at `server`: object would be replaced by number

Shallow merge

The default is a deep merge. --shallow merges top-level keys only: a later layer's value replaces the earlier one whole, so keys it omits are dropped. These are jq's two object operators:

Arrays replace wholesale in both, exactly as in jq; nothing is ever concatenated. --set layers are ordinary layers, so --shallow --set db.host=x leaves db with nothing but host.

Variable and environment references

A merged config often wants to refer to itself, or to the environment. --interpolate resolves ${key.path} and ${env:VAR} in string values, in one pass over the merged document:

# base.toml
root     = "/srv"
data_dir = "${root}/data"
port     = "${env:PORT}"
url      = "http://localhost:${env:PORT}/health"
literal  = "$${NOT_A_REF}"

$ PORT=8080 knf base.toml --interpolate
root = "/srv"
data_dir = "/srv/data"
port = 8080
url = "http://localhost:8080/health"
literal = "${NOT_A_REF}"

It is opt-in, and off by default. knf sits directly upstream of tools whose own syntax is ${...} — compose files, GitHub Actions workflows, Helm charts, systemd units. Eating those without being asked would be silent corruption, so without the flag the output is byte for byte what it is today.

Where the reference sits decides what it yields:

An environment variable is typed by the same rule as --set's right-hand side when it is the whole string, and spliced as raw text when it is embedded — parsing it only to print it again could only lose something.

$$ is a literal $. A $ followed by anything else is ordinary text, so USD $5 needs no escaping.

Document references resolve transitively and in any order; environment values are terminal and are never re-scanned. Cycles are an error, and so is a reference that names nothing:

$ knf base.toml --interpolate
error: unresolved reference
  --> server.url: `db.hostname`
  --> tags[0]: `env:REGION`
help: `${key.path}` names a key in the merged document, `${env:NAME}` an environment variable
help: drop --interpolate to pass `${...}` through untouched

A reference may also read an array element — ${servers[0].host} — with the same two-position rules: whole-string it takes the element's value and type, embedded it stringifies.

Two limits worth knowing:

• env: is a reserved prefix, matched literally rather than by splitting on the first :. So ${a:b} is the ordinary key a:b, and only keys that literally begin env: are unaddressable.
• A key spelled with brackets is unaddressable — ${a[0]} now reads as the first element of a, never as a key literally named a[0], and --set 'a[0]=1' is an error rather than a write into an array. Only a file can carry such a key. The same accepted loss as keys containing a literal dot, which the dotted grammars have always excluded.

--set layers interpolate like any other layer. --strict runs during the merge, before any substitution, so it compares the types values had when they were written.

Caveats with formats

JSON and TOML, inferred from the file extension. --input-format overrides it for every input and is required for - (stdin).

Output is the inputs' format when they agree; when they don't, -f is required rather than guessed, so reordering arguments can never silently change the encoding. Pretty-printed by default; --compact opts out.

A TOML datetime is a distinct type all the way through the merge, so every TOML output keeps it unquoted — including a merge that mixed in a JSON layer, and including --set on top. It becomes a plain string only under -f json, where there is nothing else it could be.

TOML cannot represent null, so emitting TOML from a document containing one is an error that names every path:

$ knf base.toml override.json -f toml
error: cannot serialize null to TOML
  --> servers.primary.proxy
  --> logging.sink
help: emit JSON with -f json, substitute with --null-as, or remove the null

Alternatively, you may use --null-as <string> to parse nulls into a custom value:

knf base.toml override.json -f toml --null-as=none

The option is a no-op for JSON output.

Two more values have no spelling in one format or the other, and both are rejected the same way — named by path, never silently substituted.

TOML integers are signed 64-bit, so an ID above i64::MAX (a snowflake, a hash) round-trips exactly through JSON but cannot be written as TOML at all:

$ knf ids.json -f toml
error: cannot serialize integer to TOML
  --> id: `10000000000000000001`
help: TOML integers are signed 64-bit; emit JSON with -f json

Conversely, TOML's number grammar has inf, -inf and nan literals and JSON's has none of them:

$ knf limits.toml -f json
error: cannot serialize non-finite number to JSON
  --> timeout: `inf`
help: emit TOML with -f toml, which can represent inf and nan

Each format is the escape from the other's rejection, and no same-format round-trip is affected: knf ids.json -f json and knf limits.toml -f toml both emit their input unchanged.

Testing

cargo test --workspace
cargo test -p knf-core --lib     # fast inner loop: unit tests only

License

MIT — see LICENSE.