fpr-ff1

A small, correct Python implementation of FF1, the format-preserving encryption mode from NIST SP 800-38G.

This package is intentionally just the algorithm: no accounts, no network, no key management, and no FF3/FF3-1 modes.

Install

pip install fpr-ff1

Quick start

from fpr_ff1 import FF1

# The all-zero key here is for the example only. Never use it (or any other
# published key) for real data: load key material from your secret store.
key = load_key_from_your_secret_store()  # 16, 24, or 32 bytes

ff1 = FF1(
    key=key,
    radix=10,
    alphabet="0123456789",
    # A tweak separates ciphertexts across contexts: two records with the
    # same plaintext encrypt to the same ciphertext under the same tweak,
    # so derive the tweak from stable record context (an account ID, a
    # table name) rather than leaving it empty.
    tweak=b"customer-pans",
)

encrypted = ff1.encrypt("123456")
decrypted = ff1.decrypt(encrypted)
assert decrypted == "123456"

Security notes — read before use

FF1 is a deterministic permutation for a fixed key and tweak. That has operational consequences:

1. Equal plaintexts produce equal ciphertexts under the same key and tweak. NIST recommends varying the tweak with each encryption instance where feasible — derive it from stable record context so identical plaintexts in different records encrypt differently.
2. FF1 provides confidentiality only — no integrity or authentication. Modified ciphertext decrypts to another plausible in-domain value; there is no error, and the tampering is invisible. Applications needing tamper detection must authenticate the ciphertext and its context separately, where their format allows.
3. A wrong key or tweak does not raise. Decryption with the wrong key or tweak yields plausible-looking plaintext, not an exception. There is no way to detect key mismatch from the output alone.
4. The one-million-value domain is a standards floor, not a guarantee. radix ** minlen >= 1_000_000 rules out trivially enumerable domains, but a determined attacker with oracle access can still search a million-value space. Small-domain FPE needs rate limiting or access control around the encryption interface.
5. Validation exceptions never echo your data. Rejected values are located by index, not repeated in the message, so a malformed record does not leak plaintext into your logs.

Features

• Pure Python with a single runtime dependency: cryptography.
• An optional compiled backend (backend="rust") for high-throughput callers, with the pure-Python implementation retained as the reference and the default.
• Conformance-tested against the NIST SP 800-38G sample vectors.
• No floating-point arithmetic in the FF1 core.
• Tightened domain limits from the SP 800-38G Rev. 1 second public draft:
  • radix range 2 <= radix < 2**16 (a deliberate supported subset of the spec's inclusive [2..2**16] — see Domain limits)
  • minimum domain radix ** minlen >= 1_000_000
  • maximum length 2 ** 32 - 1 (SP 800-38G specifies maxlen < 2 ** 32)
  • AES keys of 128, 192, or 256 bits only
• Strongly typed public API with typed exceptions rooted at FF1Error.

Why you can trust this implementation

Format-preserving encryption is unusually easy to get almost right. A subtly wrong FF1 still round-trips perfectly — decrypt(encrypt(x)) == x — while producing ciphertext no conformant implementation can read. By the time anyone notices, the data is written. So conformance here is not a checkbox; it is the entire product, and it is evidenced rather than asserted.

The full suite — NIST sample vectors, per-round intermediate-value conformance for every round of every sample, differential tests against an independent implementation, exhaustive bijectivity sweeps, and a malformed-input sweep — runs in CI with 100% line and branch coverage enforced. The build fails below it.

That coverage figure measures the Python package. The compiled backend's Rust core is not line-coverage measured: it is held to the same bar by running the entire conformance suite, bit-exact, against it as well, plus its own cargo test unit tests.

This is strong conformance evidence, not proof. It has not received an independent cryptographic audit or NIST validation; see What this is not below.

Conformance is verified at the round level, not just the output level

All nine published NIST sample vectors pass in both directions. That alone is a weak statement: nine input/output pairs can be satisfied by two bugs that cancel out.

So this package also asserts the per-round intermediate values the NIST sample document publishes — P, Q, R, S, y, m, c and C, plus the derived u, v, b and d — for every round of every sample, 90 rounds in total. Compensating bugs survive an output test. They do not survive this one.

The vectors are transcribed from the NIST document and stored as data files. They are never regenerated from this implementation, which would make them a record of whatever the code does rather than of what the standard requires.

Radices without published vectors are verified against an independent implementation

NIST publishes vectors for radix 10 and 36 only. Every other radix has none, so agreement with an independent implementation is the only correctness evidence available — expected values authored from this code would test nothing and lock in any bug permanently.

fpr-ff1 is therefore differential-tested against ubiq_security_fpe across radices 2, 10, 16, 32, 36, 62, 256 and 65535, including every length where the algorithm's internal block structure changes. The oracle is itself validated against all nine NIST vectors before a single comparison is trusted. Oracle-derived known-answer vectors are also frozen into the repository, so this evidence survives even if the (deprecated, unmaintained) oracle package one day stops installing.

Bijectivity is tested exhaustively for these two domains

For two domains small enough to enumerate completely — radix 2 at length 20 (1,048,576 values) and radix 10 at length 6 (1,000,000 values) — every point is encrypted and the image checked to be the full domain, with no gaps and no collisions. That is the strongest correctness statement available for a permutation, and it is run in CI rather than kept as a manual check.

The known failure modes are tested for by name

Published FF1 bugs cluster in a few places. Each has a dedicated test:

Migration is safe by construction

Output is byte-identical to ubiq_security_fpe, verified in both directions — old ciphertext decrypts with this library, and new ciphertext decrypts with the old one. Existing encrypted data stays readable, and a rollback strands nothing. See Migrating.

What this is not

Passing the published sample vectors is conformance evidence, not FIPS validation. This package is not FIPS 140 validated and makes no such claim. It also does not attempt key zeroization, and offers no constant-time guarantee — see SECURITY.md for the full statement of limitations.

Why FF1 only — and why FF3 is excluded

SP 800-38G originally specified two modes, FF1 and FF3. FF3 was revised to FF3-1 after an attack on the original construction, but Beyne subsequently demonstrated a weakness in the tweak schedule that affects FF3 and FF3-1 alike — the repair did not address the underlying problem.

The February 2025 second public draft of SP 800-38G Rev. 1 removes FF3 entirely, leaving FF1 as the only approved format-preserving mode.

fpr-ff1 will therefore never implement FF3 or FF3-1. This is a deliberate feature, not an omission: there is no configuration flag, no opt-in, and no plan to add one. If you need FF3 you need a different library, and you should first satisfy yourself that you actually need a mode NIST has withdrawn.

Domain limits are stricter than the 2016 text

This package implements SP 800-38G (2016, updated 2019) as the normative algorithm, but enforces the tightened constraints from the Rev. 1 second public draft:

The radix bound is an implementation limit, not a spec deviation: NIST permits an implementation to support a subset of radices, and this package supports 2..65535. Radix 65536 is excluded deliberately (its numerals do not fit in uint16-sized values, and no practical alphabet reaches it); if that ever changes, widening the accepted domain without changing existing behaviour will be a minor version, not a major one.

The minimum-domain rule is the one that will bite. A domain of only 100 values is trivially enumerable, so this package fails closed and rejects it. Concretely, min_length is 6 for radix 10 and 4 for radix 36 — inputs shorter than that raise LengthError, even though some older libraries (including ubiq_security_fpe) accept them.

Rev. 1 is still a draft. If it is finalised with different limits, that will be a breaking change and a major version.

Scope

• In scope: FF1 encryption and decryption, numeral and string interfaces, alphabet handling, parameter validation, tests, documentation.
• Out of scope, permanently: FF3/FF3-1, identifier generation, persistence, checksums, key generation/storage/derivation, application-specific defaults or alphabets.

Supported Python versions

3.12, 3.13 and 3.14 are the versions exercised in CI on Linux, macOS and Windows (stated in the trove classifiers). requires-python is >=3.12 with no upper bound: a capped requires-python becomes a hard resolution failure on future interpreters — a claim that they don't work, which cannot be known in advance — so the floor rises as the CI matrix grows rather than the ceiling punishing users of new Pythons.

Roadmap

The 2.0 backend is opt-in and additive: the pure-Python path is unchanged and remains the default, so existing callers are unaffected. The accelerated path is only worth having once the reference implementation is settled and there is a conformance suite strong enough to prove the two agree bit for bit — which is the point of the differential and interoperability tests, and which the 2.0 suite runs against both backends. The compiled backend removes the per-call overhead that dominates short inputs and shares 1.1's subquadratic conversion for long ones; see Backends for the measured numbers.

Nothing in the roadmap changes the scope boundary above. FF3 and FF3-1 remain permanently out of scope, and no release will add key management.

Performance

Measured on one core, CPython 3.12.13, macOS (Apple Silicon) — reproduce on your own hardware with just bench (benchmarks/timing.py is the harness):

The per-numeral cost is now flat across input lengths. Before 1.1.0 the internal conversion between a numeral sequence and a big integer was a digit-at-a-time loop, which is quadratic in the number of numerals — that was an implementation choice, not an algorithmic invariant, and 1.1.0 replaced it with subquadratic divide-and-conquer conversion plus an O(n) fast path for power-of-two radices (about 19× faster at n=20,000 radix 10, 25× at radix 256), with ciphertext bit-identical to 1.0.0 for every valid input. If you are sizing a nightly job over millions of rows, measure with just bench against production-representative hardware.

Backends

FF1 accepts a keyword-only backend parameter: "python" (the default, and the reference implementation) or "rust" (the opt-in compiled backend). Both produce bit-identical ciphertext and raise identical exceptions — validation runs in Python for both, so the typed errors and messages are the same. The compiled backend ships as fpr_ff1._rs inside the platform wheels; the pure-Python wheel and the sdist omit it, and requesting backend="rust" there raises a clear BackendError rather than an opaque ImportError.

Where the compiled backend is available. Native wheels are published for these platforms, each built once for the stable ABI (abi3) and so installable on CPython 3.12, 3.13 and 3.14:

Every one of these wheels is installed and tested on its own platform, on all three Python versions, before release. Everywhere else, including Linux with a glibc older than 2.34, musl distributions such as Alpine, other architectures, and free-threaded CPython builds, pip installs the pure-Python wheel instead. The default backend then works unchanged, and only backend="rust" raises BackendError. No native support is implied beyond the table.

Measured on one core, CPython 3.12.13, Linux x86_64 (AMD Ryzen AI Max+ PRO 395), extension built in release mode with rustc 1.98.1 — reproduce with just bench:

On every shape measured here, the compiled backend is faster. At short inputs it eliminates the per-call cipher-context construction that dominates the pure-Python path (about 55% of an n=6 call). At long inputs both cores use the same subquadratic numeral conversion — divide and conquer, plus an O(n) byte-packing path for power-of-two radices, which is why radix 256 gains most — so the compiled core keeps its lead instead of being overtaken, as it was in 2.0.0rc1.

These are one machine's numbers, not a guarantee: the ratio depends on the interpreter, the CPU and the shape of your data, so measure your own inputs with just bench before choosing.

The two backends are complementary, not a replacement: the pure-Python path remains the reference and the default, needs no compiled extension, and produces bit-identical ciphertext.

API

FF1(key, radix, *, alphabet=None, tweak=b"", min_tweak_len=None, max_tweak_len=None, backend="python")

The package exports fpr_ff1.__version__ — the version of the installed distribution. Callers recording which build produced a dataset should capture it alongside their data.

Instances are picklable and deep-copyable (the cipher objects are rebuilt on the far side), so an FF1 can be passed to multiprocessing workers or broadcast by PySpark. Note that pickling an instance serialises the key — see SECURITY.md.

Numeral interface

The primitive interface works on integers in [0, radix).

ciphertext = ff1.encrypt_numerals([1, 2, 3, 4, 5, 6])
plaintext = ff1.decrypt_numerals(ciphertext)

Accepted numeral types. Anything losslessly integral — int, IntEnum, and integers from other numeric libraries such as NumPy, which are normalised to Python int so fixed-width values cannot overflow in the internal big-integer arithmetic.

float, Decimal, Fraction and str are rejected with ValueRangeError, even when they compare equal to a valid numeral: 1.0 < 10 is True, so comparison alone is not a type check.

bool is rejected deliberately. True would otherwise encrypt silently as 1, and a list of booleans arriving here is a caller mistake, not an intent to encrypt ones and zeros.

The input must be a Sequence — something with a known length. A generator raises TypeError (not FF1Error), because that is misuse of the API rather than bad data; wrap it in list(...). The Sequence contract is enforced: mappings and sets are rejected, and a Sequence whose __len__ disagrees with the values it yields raises LengthError rather than encrypting a domain smaller than the enforced minimum.

String interface

When alphabet is provided, the string interface maps characters to numerals and back.

ff1.encrypt("123456")
ff1.decrypt("654321")

Alphabet uniqueness is by Unicode code point. FF1 operates on code points, so normalisation is the caller's responsibility. Precomposed é (U+00E9) and decomposed é (U+0065 U+0301) are visually identical but count as two distinct symbols, and an alphabet containing both is accepted. If your alphabet comes from user input or an external source, normalise it first:

import unicodedata

alphabet = unicodedata.normalize("NFC", alphabet)

Exceptions

Every rejection raises a typed exception derived from FF1Error. Nothing is silently truncated, padded, coerced or clamped.

AlphabetError signals malformed configuration (caught at construction); ValueRangeError signals malformed data (caught per call). They are deliberately distinct so callers can handle a programming error differently from a bad input record.

Thread safety

FF1 instances are thread-safe. No mutable state is shared between calls — every cipher context is created locally to the call that uses it — so separate calls on one instance may run concurrently and produce exactly the single-threaded results. There is no module-level or global state either, so any number of instances may be used concurrently. A web service may freely share one FF1 across request threads.

Thread-safe is not the same as parallel. The pure-Python backend holds the GIL throughout, so concurrent calls interleave rather than overlap. The compiled backend releases the GIL for the duration of the FF1 computation, so concurrent calls on one instance genuinely run in parallel — measured 2.9× on four threads (n = 5,000, radix 10) against 0.96× for the pure-Python control. Releasing the GIL also means a long call no longer stalls unrelated threads in the process. Reproduce both rows with just bench.

Migrating from ubiq_security_fpe

fpr-ff1 exists to replace ubiq_security_fpe, which was deprecated in favour of a SaaS client and is no longer maintained. The two produce identical ciphertext for identical inputs, so existing encrypted data stays readable — no re-encryption, no migration window, no rollback risk.

That claim is enforced by tests/test_interoperability.py, which checks both directions (old ciphertext decrypts with the new library and vice versa) across all three key sizes, tweaked and untweaked. Migration safety is treated as a correctness obligation, not a promise.

No compatibility shim ships, deliberately. A Context(...) / .Encrypt() drop-in would mean maintaining a permanent second API, in a naming style this project does not use, mirroring a library that is itself deprecated. The migration below is three mechanical edits per call site, and the part that would actually be hard — identical ciphertext — is already done.

API mapping

# before
from ubiq_security_fpe import ff1

ctx = ff1.Context(key, tweak, twk_min_len, twk_max_len, radix, alphabet)
ciphertext = ctx.Encrypt(plaintext, None)
plaintext = ctx.Decrypt(ciphertext, None)

# after
from fpr_ff1 import FF1

ctx = FF1(
    key, radix, alphabet=alphabet, tweak=tweak, min_tweak_len=twk_min_len, max_tweak_len=twk_max_len
)
ciphertext = ctx.encrypt(plaintext)
plaintext = ctx.decrypt(ciphertext)

Behaviour changes to check before you switch

1. Shorter inputs are rejected. fpr-ff1 enforces the Rev. 1 draft's radix ** minlen >= 1_000_000; ubiq_security_fpe enforced the same rule, so ciphertext produced by the legacy library decrypts unchanged. But if your data contains values that only ever passed under the 2016 text's weaker >= 100 bound — through another library or a manual path — those inputs now raise LengthError. Before switching, check your shortest values against ctx.min_length for your radix (6 for radix 10, 4 for radix 36, 3 for radix 256).
2. Errors are typed. Rejections raise KeyLengthError, RadixError, LengthError, ValueRangeError, TweakLengthError or AlphabetError — all subclasses of FF1Error — rather than bare RuntimeError. Catch FF1Error if you want the old catch-all behaviour.
3. No M2Crypto dependency. fpr-ff1 depends only on cryptography.
4. Alphabet is validated at construction. A wrong-length alphabet or one with duplicate characters raises AlphabetError immediately rather than misbehaving later.

FIPS disclaimer

Passing the published NIST sample vectors is evidence of conformance. It is not FIPS validation. This package makes no claims of FIPS 140 conformance.

Key material

fpr-ff1 does not attempt to zeroize key material. Python bytes are immutable and the interpreter may copy them during garbage collection.

Development

Requires Python 3.12, uv, and just.

just setup    # create venv and install deps
just quality  # format check, lint, typecheck, tests
just build    # quality gate + uv build
just secrets  # gitleaks scan (must be installed locally)

Documentation

• docs/architecture.md — design and module overview
• docs/developer-guide.md — setup, commands, testing, and CI
• docs/directory-structure.md — repository layout
• docs/configuration.md — FF1 constructor parameters and runtime constraints
• docs/backlog.md — active and completed work
• CHANGELOG.md — release history, including behaviour changes that affect accepted inputs
• SECURITY.md — disclosure process and known limitations
• CONTRIBUTING.md — how to contribute, including the vector-provenance rules
• CODE_OF_CONDUCT.md — community standards

License

MIT