libfte

Format-Transforming Encryption: encrypt data so the ciphertext matches any format you specify.

What is FTE?

Unlike standard encryption that produces random-looking output, FTE produces ciphertext that looks like whatever format you specify (via a regular expression, or any RankedFormat provider you supply), so it can look like hex strings, alphanumeric tokens, any language a regex can denote, or a custom format of your own.

One engine, two axes. fte.FTE maps rank_in -> transform -> unrank_out over an input_format / output_format pair (the input defaults to raw bytes) and a cipher: "aes-ctr-hmac" (randomized, authenticated) or a deterministic cipher object. FPE is the equal-formats case; classic FTE is the bytes-input case. fte.RegexFormat is the built-in provider; supply your own RankedFormat for any other language. The wire format changed in 0.4.0 and is not compatible with libfte 0.3.x and earlier.

Installation

pip install fte

libfte itself is pure Python. It depends on cryptography (AES-CTR on OpenSSL; it ships prebuilt wheels with OpenSSL bundled), regex2dfa (pure Python) and libffx (pure Python; the FF1 format-preserving cipher), so no compiler or system library is needed on the platforms cryptography publishes wheels for.

Quick Example

Encrypt a secret so the ciphertext looks like words:

import os
import fte

key = os.urandom(32)  # 32 bytes, shared by both endpoints

# Pick a covertext format, then build a cipher over it and the key.
# 73 characters of words hold up to 15 plaintext bytes (cipher.max_plaintext_bytes).
word_format = fte.RegexFormat(r'^([a-z]+ )+[a-z]+$', length=73)
cipher = fte.FTE(output_format=word_format, key=key)

ciphertext = cipher.encrypt(b'Attack at dawn')
print(ciphertext.decode())
# One real run; the exact text varies per call, because the cipher is randomized:
# aa migbcjfbkvhczkjjwogvkpr m hnczwlthnujcutvnxqtrfhfnvnjhowaax mg nazfkrf

plaintext = cipher.decrypt(ciphertext)
# → b'Attack at dawn'

The covertext is a string of the chosen format that carries your encrypted message. Because the format holds one byte more than the message needs, the covertext can begin with a short run of the format's lowest-ranked symbols (a and space); a much larger length would make that run long.

RegexFormat also takes a min_length/max_length range for variable-length covertext; a fixed length is the special case where they are equal.

Format-preserving and deterministic FTE. cipher="ff1" (NIST SP 800-38G FF1 via libffx) is deterministic and zero-expansion: pass the same format as input_format and output_format to re-encrypt a value in place (FPE, length preserved), or two different formats for a deterministic rank map between them. It refuses an input domain below one million values, is unauthenticated, and leaks plaintext equality, so pass per-record tweak values and never reuse a key across the two ciphers.

Ranked-Format Providers

FTE accepts any object implementing the structural RankedFormat protocol: reversible rank() and unrank() methods. Providers need no inheritance, registration, or runtime dependency on libfte:

import secrets

import fte


class DecimalText:
    def rank(self, value: str, /) -> int:
        if not value.isascii() or not value.isdigit():
            raise ValueError("not canonical decimal text")
        if value != "0" and value.startswith("0"):
            raise ValueError("not canonical decimal text")
        return int(value)

    def unrank(self, index: int, /) -> str:
        if type(index) is not int or index < 0:
            raise ValueError("invalid rank")
        return str(index)

shared_32_byte_key = secrets.token_bytes(32)
cipher = fte.FTE(output_format=DecimalText(), key=shared_32_byte_key)
covertext: str = cipher.encrypt(b"secret")
assert cipher.decrypt(covertext) == b"secret"

The key and exact ranked-format ordering must match at both endpoints. Generic FTE framing exposes plaintext length through the rank and guarantees membership in the format's language, not a uniform distribution over unused format capacity: a fixed-length covertext much larger than the message begins with a run of the format's lowest-ranked symbols.

Use Cases

• Protocol obfuscation: Make encrypted traffic look like benign data
• Bypassing filters: Evade systems that block encrypted-looking content
• Constrained fields: Confine ciphertext to a required character set or field shape, such as an alphanumeric account token or a fixed-width record field

Documentation

Full docs and examples: github.com/kpdyer/libfte

Reference

Based on Protocol Misidentification Made Easy with Format-Transforming Encryption (ACM CCS 2013) and LibFTE: A Toolkit for Constructing Practical, Format-Abiding Encryption Schemes (USENIX Security 2014).

License

MIT