Skip to main content

jev-secrets

Replaces names, keys and other secrets in a Jev request with stand-ins before it is sent, and puts the originals back in the answers. For the PHP SDK and the official JavaScript and Python SDKs, or any code that builds the request body itself.

John Smith <john.smith@acme.com> was charged twice on 2024-03-15. Key sk_live_4eC39HqLyjWD
Tozd Cpabl <tozd.cpabl@ulco.zuy> was charged twice on 2024-03-15. Key sk_live_3uF21GlNlxHG

The second line is what Jev sees, with John Smith passed as a value and the rest found by the default patterns. In each replaced word every letter becomes another letter of the same case, vowels stay vowels, digits become digits, and punctuation stays, so a name still reads as a name and an email address as an email address. The same word gets the same stand-in everywhere in the request, including in the questions. The answers come back with the original question ids and choice labels.

This is pseudonymisation, not anonymisation: the mapping is held in memory in order to be able to reverse the process on the way back to the caller.

Contents: Install · Use · What gets replaced · What a stand-in looks like · The key · What is rewritten and restored · Config · When it refuses to send · What it does not do · Does it change the answers?

Install

composer require phox/jev-secrets        # PHP 8.3+, with ext-intl and ext-mbstring
npm install @phox-js/jev-secrets         # Node 20.19+
pip install jev-secrets                  # Python 3.10+; jev-secrets[sdk] adds the SDK wrapper
Tested on SDK versions the wrapper is tested with
PHP 8.3, 8.4, 8.5 phox/typesafe-sdk-php 0.3 and 0.4
JavaScript Node 20.19, 22, 24 @typesafe-ai/sdk 0.5.7 and 0.6.0
Python 3.10 to 3.14 typesafe-sdk 0.7.0 and 0.7.1

A request of 1000 strings, 500 KB of JSON, takes 0.4-0.5 s in PHP and JavaScript and 1.1-1.4 s in Python; docs/performance.md has the measurements.

Use

With the SDK, wrap the client. Each call takes the values to hide in that request, on top of whatever the config finds:

use Phox\JevSecrets\Secrets;
use Phox\JevSecrets\TypeSafe\SecretClient;

$client = new SecretClient(new \Phox\TypeSafe\Client(), new Secrets(['terms' => ['Acme Ltd']]));

$response = $client->systemOne()
    ->state(['ticket' => $ticket])
    ->noul('refund', 'Does John Smith ask for a refund?')
    ->values(['John Smith'])
    ->send();
import { TypeSafeClient } from '@typesafe-ai/sdk';
import { Secrets, withSecrets } from '@phox-js/jev-secrets';

const client = withSecrets(new TypeSafeClient(), new Secrets({ terms: ['Acme Ltd'] }));
const { answers } = await client.systemOne(
    { state: { ticket }, questions: { refund: { type: 'noul', instructions: 'Does John Smith ask for a refund?' } } },
    { values: ['John Smith'] },
);
from typesafe_sdk import TypeSafeClient
from jev_secrets import Secrets
from jev_secrets.typesafe import SecretClient

client = SecretClient(TypeSafeClient(), Secrets({'terms': ['Acme Ltd']}))
result = client.system_one(
    state={'ticket': ticket},
    questions={'refund': {'type': 'noul', 'instructions': 'Does John Smith ask for a refund?'}},
    values=['John Smith'],
)

jev_secrets.typesafe.AsyncSecretClient wraps the async Python client. The wrappers sit above the SDK, not in its transport, because the SDKs log request bodies at debug level before the transport sees them.

Any config option can be changed at call time. scope and dates merge entry by entry; any other option given replaces the configured one:

$client->systemOne()->state($ticket)->noul('refund', 'Refund?')
    ->overrides(['disable' => ['phone'], 'dates' => ['order' => 'mdy']])
    ->send();

In JavaScript it is options.overrides, in Python overrides=, and without an SDK the third argument of pseudo().

Without an SDK, rewrite the body yourself and restore the decoded response:

$done = (new Secrets())->pseudo($body, ['John Smith']);
$answers = $done->restore(json_decode(send($done->request), true));

pseudo() is also available as pseudonymise(), pseudonymize() and mask(), and restore() as unmask().

$done->replacements lists what was replaced, as a path into the sent request and the source that found it, without the values themselves.

What gets replaced

The sources, all optional:

Source What it matches
Values passed to the call those strings
Config terms (fixed strings), fields (paths whose whole value is replaced), patterns (your regexes)
Defaults the patterns below, and the value under a key such as password, token or api_key

secretKeys controls those key names: false for none, a list for exactly those, or {"add": [...], "remove": [...]} to change the default list. Names compare lowercased, ignoring spaces, hyphens and underscores, so member_number covers Member-Number.

Values and terms match case-insensitively, at word boundaries. Their words may be joined by any run of spaces, underscores, hyphens or dots, so Ellen Park is also found in is_from_ellen_park and ellen.park@example.com. A value shorter than 2 characters is refused. The value found at a field path is also replaced wherever else it appears in the request.

Names cannot be found by pattern. Pass them as values, list them as terms, or name the fields they're in.

Dates, and numbers the default patterns do not cover, are not replaced by default. Pass one as a value, or name its field, and a number is replaced digit by digit, a date or time moved (below). With dates.detect: true every date and time in the request is moved.

Default pattern Finds
email an email address
ipv4 a dotted-quad IPv4 address
ipv6 an IPv6 address, full or compressed, with at least one decimal digit
card a card number of 12 to 19 digits that passes the Luhn check
iban an IBAN that passes the mod-97 check
phone a phone number written with a leading + and 8 to 15 digits
us-ssn a US Social Security number written with dashes
uk-nino a UK National Insurance number
aws-access-key an AWS access key id, keeping its prefix, such as AKIA
github-token a GitHub token, keeping its prefix, such as ghp_
github-pat a fine-grained GitHub token, keeping the github_pat_ prefix
stripe-key a Stripe secret or restricted key, keeping its prefix, such as sk_live_
slack-token a Slack token, keeping its prefix, such as xoxb-
anthropic-key an Anthropic API key, keeping its prefix, such as sk-ant-api03-
openai-key an OpenAI API key, keeping its prefix, such as sk-proj-
google-api-key a Google API key, keeping the AIza prefix
jwt a JSON Web Token, keeping the leading eyJ
private-key the body of a PEM private key, keeping the BEGIN and END lines
bearer-token a bearer token of 16 or more characters, keeping the word Bearer
credential-assignment a value after password=, api_key:, token= and the like in text, keeping the name

Where two matches overlap they merge, so a value inside an email address takes the rest of the address with it. catalogue/patterns.json holds the patterns and the key names; the PHP, JavaScript and Python implementations read the same file.

What a stand-in looks like

Recorded outputs of the character generator under a fixed key, from catalogue/fixtures.json:

Input Stand-in
John / JOHN Tozd / TOZD
O'Brien-Smythe I'Ghees-Dmjcbu
José Müller Taré Hömsif
Дмитрий Иванов Ьхэвлаӗ Ычозюв
김민준 펇쫹떀
2024-12-31 23:59 1011-11-21 12:33
10.0.0.1 22.9.9.7
  • Letters: each becomes another from the same alphabet and case. With letters: "shape", the default, vowels stay vowels and consonants stay consonants, so the stand-in is pronounceable. Latin, Greek, Cyrillic, Armenian, Georgian, Hebrew, Arabic, Devanagari, Thai, kana, Hangul and CJK are drawn from their own scripts; a letter from any other script becomes an ASCII letter. Accents stay on.
  • Digits: with digits: "lower", the default, 0 and 1 stay and every other digit becomes a lower one of at least 1, so an IP address or a version stays valid. A span made only of 0s and 1s, like 10.0.0.1, is drawn under digits: "any" instead.
  • Dates and times are moved, not rewritten: every date in the request by the same number of weeks, every time by the same number of minutes, without crossing midnight. Order, intervals and weekdays survive, and the format stays as written, so Monday, 16 March 2026 at 3:40 pm becomes Monday, 12 October 2026 at 11:46 am. dates.order decides whether 03/04/2026 is 3 April (dmy, the default) or 4 March (mdy); a year written first is always year, month, day. With dates.mode: "digits" they get the digit rule instead, which is how the 2024-12-31 23:59 row above was made. docs/dates.md lists the formats.
  • Everything else stays: spaces, punctuation, @, ., -, _.

A stand-in is drawn again until it differs from every word already in the request and from every other stand-in. Short numbers under the digit rule can run out of stand-ins and share one; docs/generation.md has the exact rules.

The key

Stand-ins and date offsets are drawn under a key:

key What each call uses
left out the JEV_SECRETS_KEY environment variable, read on every call; without it, a random key
a string that string
false a random key, whatever the environment holds

With a random key the stand-ins change from call to call. With a fixed one the same request gives the same stand-ins every time, in every implementation. Nothing is stored between calls, so rotating the key costs nothing: calls after the change get different stand-ins. The library reads the process environment, not a .env file; load that the way your framework does.

What is rewritten and restored

Part of the request Rewritten Restored in the answers
state strings and numbers yes -
state object keys with scope.keys -
question instructions and criteria text yes a score's legend, exactly as sent
question ids where they contain a match the answer keys
choice labels where they contain a match choice and the keys of probabilities
noul criteria keys, type, model, anything else at the top level never -

A score's legend keys and probability keys are rubric indices and are never restored, nor is any number in the response. A string field in an answer that this library does not know gets word-by-word replacement of the stand-ins.

Config

The same object in PHP, JavaScript and Python. spec/config.schema.json has every option and its default.

{
  "terms": ["Acme Ltd", "Project Nightingale"],
  "fields": ["state.customer.name", "state.orders.*.email"],
  "patterns": [{ "id": "order-id", "pattern": "ORD-(?<value>[0-9]{6})" }],
  "allow": ["support@acme.com"],
  "disable": ["ipv4"],
  "scope": { "keys": true },
  "secretKeys": { "add": ["member_number"] },
  "dates": { "order": "mdy", "detect": true }
}

allow protects a string from terms, fields, patterns and the defaults; a value passed to a call still replaces it.

When it refuses to send

After rewriting, every original value is looked for again across every string and key under state and questions. If one is still there, the call throws LeakException (LeakError in JavaScript and Python) and nothing is sent. The message names the path and the source, never the value, and suggests the fix for the usual causes: a value in an object key while scope.keys is off, or in the questions while scope.questions is off.

In PHP, a request that cannot be searched is refused too: text that is not valid UTF-8, or a regex that stops at PCRE's backtrack limit (MatchException). Matching cannot say what it missed, so nothing is sent. JavaScript and Python strings are always Unicode, and their regexes have no such limit.

A config that cannot work is refused when it is made: an unknown option, a pattern that does not compile, a term with no letter or digit or shorter than minLength. A value passed to a call is checked the same way when the call is made.

What it does not do

  • Find names by itself. Only values, terms and fields hide names.
  • Hide shape. A stand-in has the same length, capitals, punctuation and word count as the original, and the same word always gets the same stand-in, so the number of times a name appears is visible.
  • Hide every digit. Under the default digit rule 0 and 1 are kept and 2 can only become 1.
  • Keep order between numbers. Each number is lowered on its own, so 19 and 22 can come out as 17 and 11, and two short numbers can share a stand-in when the rule leaves no other in range. Dates and times keep their order unless dates.mode is digits.
  • Keep a date's relation to the real world. A moved date is up to a year away from the original, so "is this overdue?" against today, "last Friday" or an age written in words no longer line up.
  • Catch everything the defaults target. The patterns trade recall for precision: a phone number without a leading + is missed, a version string like 1.2.3.4 reads as an IPv4 address, and about one in ten 12 to 19 digit numbers passes the Luhn check by chance.
  • Unlink calls under a fixed key. With a fixed key, from the config or JEV_SECRETS_KEY, a word gets the same stand-in in every call, so whoever receives the requests can tell that two calls mention the same person, and anyone holding the key can test a guessed name against a stand-in. Use a random key unless you need repeatable output.
  • Keep an object keyed "0", "1", ... an object in PHP. PHP decodes it, and {}, as a list, so its keys are not rewritten and it is sent as a JSON array. The PHP SDK has the same limit. For the same reason secretKeys: {} means no secret keys in PHP, and no change to the default list in JavaScript and Python.
  • Keep numeric-keyed requests identical in JavaScript. JavaScript orders integer-like object keys ascending, and stand-ins that collide are redrawn in walk order.

Does it change the answers?

Over 67 labelled questions on 50 PII-heavy requests (docs/evidence.md):

  • The default config got 64 of 67 right in both runs; tags got 58, the plain request 67. The default moved answers least, by 0.05 on average against 0.17 for tags.
  • All 16 questions comparing dates or times kept their answers: every date and time moves by the same offset. Names in Chinese, Arabic, Greek and Japanese script made no difference.
  • What replacing removes is lost: whether an email domain is a company's, or +44 a UK number.
  • Comparing two plain numbers passed as values fails, since the digit rule keeps neither size nor order. Leave out of values any number a question compares.

Documentation

docs/generation.md and docs/dates.md are the byte-level contract the implementations share. catalogue/fixtures.json holds the cases each must reproduce: php scripts/golden.php records them from the PHP implementation, and the JavaScript and Python suites require the same output.

Release files for jev-secrets 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for jev-secrets 1.0.0
File Size Uploaded
jev_secrets-1.0.0.tar.gz 58.0 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for jev-secrets 1.0.0
File Interpreter ABI Platform
jev_secrets-1.0.0-py3-none-any.whl Python 3 none any Details

Total release size: 100.0 kB

Release files / jev_secrets-1.0.0.tar.gz

Download URL jev_secrets-1.0.0.tar.gz
Size 58.0 kB
Tags Source
SHA-256 checksum
How to use checksums
26d8fdc2d070116a1fda2b3da8609b775b9756caa4fa40003ef784d6d6ad91f1
BLAKE2b-256 checksum
How to use checksums
2fe40b257e2adf5b91a40091a6aec72d3b32f4a1064722f454bb01f089e4f9c9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.18 {"installer":{"name":"uv","version":"0.12.18","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / jev_secrets-1.0.0-py3-none-any.whl

Download URL jev_secrets-1.0.0-py3-none-any.whl
Size 42.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7a83c9ae167310b41d7e35a6a5e4c482343de62ce3c0346aca94a0964827ef3b
BLAKE2b-256 checksum
How to use checksums
ebbe0cfb957162003664701f9fb11067b1cfb5c6a2839dbfc2869e93462d3831
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.18 {"installer":{"name":"uv","version":"0.12.18","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release history Release notifications | RSS feed

This release

1.0.0 This release

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page