Skip to main content

Japanese PII masking for LLM pipelines

Project description

kakushi 隠し

Japanese PII masking for LLM pipelines. Strip personal names, company names, phone numbers, emails, amounts, and my-numbers from Japanese business text before it leaves your server. Restore them in the model reply.

Live demo on HuggingFace Spaces · 日本語の説明


Why

You want to send a Japanese email, meeting note, or invoice to OpenAI / Anthropic / Gemini for summarisation, rewriting, or QA. The text contains customer names (田中太郎), company names (株式会社山田商事), deal amounts (¥1,500,000), and contact details. You don't want those bytes leaving your server.

kakushi gives you a three-line workaround.

from kakushi import mask, unmask

masked, mapping = mask(
    "田中太郎様、株式会社山田商事の件で¥1,500,000の見積書を tanaka@example.co.jp にお送りします",
    dictionary={"株式会社山田商事": "取引先A"},   # optional — pin sensitive names
)
# masked  → "<PERSON_1>様、取引先Aの件で<AMOUNT_1>の見積書を <EMAIL_1> にお送りします"
# mapping → in-memory only. Never persisted.

reply = call_openai(masked)                   # your LLM call here
final = unmask(reply, mapping)                # placeholders → originals

Install

pip install kakushi          # regex layer only — fast, no model download
pip install kakushi[ner]     # + GiNZA Japanese NER (person / org / location)

What gets caught

Three layers, lowest priority first.

  1. GiNZA NERPERSON, ORG, LOCATION from the Japanese spaCy pipeline.
  2. Regex — phone, postal code, email, my-number, yen amount, IPv4, driver's license, credit card, URL, contract / order numbers, and Japanese company names with 株式会社 / 合同会社 / (株) markers.
  3. Company dictionary (optional){"株式会社山田商事": "取引先A"}. Highest priority. The value can be None, in which case kakushi auto-assigns <COMPANY_DICT_N>.

Overlapping spans are resolved with priority dict > regex > NER, with the longer span winning at the same priority.

Recall on synthetic corpus (100 cases, NER on)

Entity Recall
AMOUNT 100%
EMAIL 100%
PERSON 100%
PHONE 100%
ORG 92%
LOCATION 60.9%
Overall 95.7%

Reproduce with python scripts/eval.py --n 100.

Known limitations

  • Hiragana-leading company names (株式会社さくら通信, 合同会社みどり) are missed by the regex because including hiragana in the body would destroy the particle boundary. Register them via the company dictionary.
  • Compound place names like 京都府京都市 are sometimes split by GiNZA.
  • My-number checksums are not validated yet. Any 12-digit dddd-dddd-dddd pattern is matched.
  • The mapping returned from mask() is in-memory only. Do not serialize it, log it, or write it to disk. It contains the originals you just removed.

Try it without writing code

The HuggingFace Space lets you paste any Japanese text, optionally provide a company dictionary, and see exactly what would be sent to the LLM. Nothing is logged.

Project layout

kakushi/
├── src/kakushi/
│   ├── normalizer.py     # NFKC + context-aware hyphen folding
│   ├── patterns.py       # 12 regex patterns (PII + Japanese company names)
│   ├── ner.py            # GiNZA wrapper (lazy load, optional)
│   ├── resolver.py       # span overlap resolution
│   ├── masker.py         # mask() / unmask() / company dictionary
│   ├── synth.py          # synthetic business-text generator
│   └── types.py          # EntityType, Match
├── tests/                # 70 cases, all green
├── scripts/eval.py       # recall measurement on synthetic corpus
├── examples/openai_demo.py
└── space/                # HuggingFace Space app (Gradio)

License

MIT.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kakushi-0.1.1.tar.gz (22.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

kakushi-0.1.1-py3-none-any.whl (15.1 kB view details)

Uploaded Python 3

File details

Details for the file kakushi-0.1.1.tar.gz.

File metadata

  • Download URL: kakushi-0.1.1.tar.gz
  • Upload date:
  • Size: 22.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for kakushi-0.1.1.tar.gz
Algorithm Hash digest
SHA256 839ba1e94e7628f95bb029f44db7268ff4b0d11ef60d728cae667d7d516321ea
MD5 8d5f7cac6697cd1b706823f8dbd3455e
BLAKE2b-256 01cd6a2f0f0e8c4c999aaf6d439ffd655ede3904a3032473de3882f7d052ac03

See more details on using hashes here.

File details

Details for the file kakushi-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: kakushi-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 15.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for kakushi-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 fa974cd787d31e54240182cf0bbf42ab62f78ce670015e14e9550d3fa3aaf65d
MD5 d67a570baecbd0e81f40583533ddbeda
BLAKE2b-256 8c77bc40b71d945227d58cf74f62312a946da6a3c2405da4a4461605aa62ae77

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page