Skip to main content

Japanese PII masking for LLM pipelines

Project description

kakushi 隠し

Japanese PII masking for LLM pipelines. Strip personal names, company names, phone numbers, emails, amounts, and my-numbers from Japanese business text before it leaves your server. Restore them in the model reply.

Live demo on HuggingFace Spaces · 日本語の説明


Why

You want to send a Japanese email, meeting note, or invoice to OpenAI / Anthropic / Gemini for summarisation, rewriting, or QA. The text contains customer names (田中太郎), company names (株式会社山田商事), deal amounts (¥1,500,000), and contact details. You don't want those bytes leaving your server.

kakushi gives you a three-line workaround.

from kakushi import mask, unmask

masked, mapping = mask(
    "田中太郎様、株式会社山田商事の件で¥1,500,000の見積書を tanaka@example.co.jp にお送りします",
    dictionary={"株式会社山田商事": "取引先A"},   # optional — pin sensitive names
)
# masked  → "<PERSON_1>様、取引先Aの件で<AMOUNT_1>の見積書を <EMAIL_1> にお送りします"
# mapping → in-memory only. Never persisted.

reply = call_openai(masked)                   # your LLM call here
final = unmask(reply, mapping)                # placeholders → originals

Install

pip install kakushi          # regex layer only — fast, no model download
pip install kakushi[ner]     # + GiNZA Japanese NER (person / org / location)

What gets caught

Three layers, lowest priority first.

  1. GiNZA NERPERSON, ORG, LOCATION from the Japanese spaCy pipeline.
  2. Regex — phone, postal code, email, my-number, yen amount, IPv4, driver's license, credit card, URL, contract / order numbers, and Japanese company names with 株式会社 / 合同会社 / (株) markers.
  3. Company dictionary (optional){"株式会社山田商事": "取引先A"}. Highest priority. The value can be None, in which case kakushi auto-assigns <COMPANY_DICT_N>.

Overlapping spans are resolved with priority dict > regex > NER, with the longer span winning at the same priority.

Recall on synthetic corpus (100 cases, NER on)

Entity Recall
AMOUNT 100%
EMAIL 100%
PERSON 100%
PHONE 100%
ORG 92%
LOCATION 60.9%
Overall 95.7%

Reproduce with python scripts/eval.py --n 100.

Known limitations

  • Hiragana-leading company names (株式会社さくら通信, 合同会社みどり) are missed by the regex because including hiragana in the body would destroy the particle boundary. Register them via the company dictionary.
  • Compound place names like 京都府京都市 are sometimes split by GiNZA.
  • My-number checksums are not validated yet. Any 12-digit dddd-dddd-dddd pattern is matched.
  • The mapping returned from mask() is in-memory only. Do not serialize it, log it, or write it to disk. It contains the originals you just removed.

Try it without writing code

The HuggingFace Space lets you paste any Japanese text, optionally provide a company dictionary, and see exactly what would be sent to the LLM. Nothing is logged.

Project layout

kakushi/
├── src/kakushi/
│   ├── normalizer.py     # NFKC + context-aware hyphen folding
│   ├── patterns.py       # 12 regex patterns (PII + Japanese company names)
│   ├── ner.py            # GiNZA wrapper (lazy load, optional)
│   ├── resolver.py       # span overlap resolution
│   ├── masker.py         # mask() / unmask() / company dictionary
│   ├── synth.py          # synthetic business-text generator
│   └── types.py          # EntityType, Match
├── tests/                # 70 cases, all green
├── scripts/eval.py       # recall measurement on synthetic corpus
├── examples/openai_demo.py
└── space/                # HuggingFace Space app (Gradio)

License

MIT.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kakushi-0.1.0.tar.gz (23.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

kakushi-0.1.0-py3-none-any.whl (15.1 kB view details)

Uploaded Python 3

File details

Details for the file kakushi-0.1.0.tar.gz.

File metadata

  • Download URL: kakushi-0.1.0.tar.gz
  • Upload date:
  • Size: 23.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for kakushi-0.1.0.tar.gz
Algorithm Hash digest
SHA256 9fcc08b096d5ae57dfa7a4b15925b3aacf7a4f71a8434ba4f89c031517201c07
MD5 4b6ef623c3f03cc6df5030186be064ba
BLAKE2b-256 1a697d75a820055f8761c2b5c42ee5bd5aaac0a6be79d8963f48b88d125175d9

See more details on using hashes here.

File details

Details for the file kakushi-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: kakushi-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 15.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for kakushi-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 2e324efdbf862326551b8984c1abb491a731888361b12598f2dcc558873498d7
MD5 85945c37bcbd11a0a514bdea0de8ed9a
BLAKE2b-256 f4cd65382fa750cde94cae9fa39933ed6e64027e99911b9fccbe778efd589a9d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page