language: ko license: other license_name: schift-2.0 license_link: LICENSE library_name: transformers pipeline_tag: token-classification tags:

• pii
• korean
• ner
• privacy
• token-classification datasets:
• custom model-index:
• name: schift-ko-pii-v4 results:
  • task: type: token-classification name: Korean PII Detection metrics:
    • type: f1 value: 0.863 name: F1 (raw, no postprocess)
    • type: f1 value: 0.968 name: F1 (with postprocess)

schift-ko-pii-v4

111M parameter Korean PII detector built on klue/roberta-base.

Fine-tuned with LoRA on Korean legal, financial, and administrative texts. The active service artifact is the v4 ONNX package.

Benchmark status

The active release remains v4. The dated report records the v5 candidate comparison and decision: docs/2026-08-18-pii-v5-latency-cost.md.

Release decision (2026-08-19)

HOLD — v4 remains active; v5 is a measured candidate, not the active release. The v5 candidate scored F1 0.629 raw and 0.929 with postprocessing on the local 93-case suite, and F1 0.8906 on the 100-case hard holdout. Its measured CPU normal-call latency was 13.74x v4 on average, so the quality result does not yet justify the latency/cold-start cost. The active Cloud Run service and the checked-in dist package still reference v4. No v5 release pointer or deployment is implied by this README.

The complete evidence, exact commands, artifact commit, package metadata, and service revision are recorded in ../docs/2026-08-18-pii-v5-latency-cost.md.

Quick start

pip install schift-ko-pii

from schift_ko_pii import detect

# Raw model output
spans = detect("피고 김민수의 전화번호는 010-1234-5678이다.")
# [
#   {"start": 3, "end": 6, "label": "private_person", "score": 0.99, "text": "김민수"},
#   {"start": 14, "end": 27, "label": "private_phone", "score": 0.98, "text": "010-1234-5678"},
# ]

# With postprocessing (regex validation + structured-ID rules, higher precision)
spans = detect("주민번호 850205-1234567을 확인.", postprocess=True)

Postprocessing

The postprocess=True flag applies Korean-specific rules:

• Regex validation for structured IDs (resident numbers, passports, vehicle plates, bank accounts)
• Luhn/checksum verification where applicable
• Context-aware span merging
• False-positive suppression for legal case numbers and statute references

This raises strong-category F1 from 0.893 to 0.968.

API (free)

For production use without managing model files:

from schift import Schift

client = Schift(api_key="...")  # free at schift.io
result = client.pii.redact("김민수의 전화번호는 010-1234-5678입니다.")
# Postprocessing is always enabled on the API.

Labels

Benchmark

The benchmark suite is included in this repository under benchmark/.

# Benchmark this model
python benchmark/run_benchmark.py

# With postprocess
python benchmark/run_benchmark.py --postprocess

# Compare any HuggingFace model
python benchmark/run_benchmark.py --hf-model LiquidAI/LFM2.5-Encoder-350M-PII-Detector

93 hand-crafted cases covering Korean legal, investigative, and administrative text. Categories: person (standard/rare/short/foreign/hanja), phone, email, address (urban/rural), dates, URLs, resident IDs, business IDs, bank accounts, passports, vehicle plates, IP addresses, plus negative cases (legal case numbers, statute references, Hanja legal terms).

Model details

• Base model: klue/roberta-base (111M params)
• Training: LoRA adapter on Korean legal/financial/admin examples
• Format: v4 ONNX service artifact / safetensors release source
• Inference: transformers pipeline, CPU or GPU
• Max length: 512 tokens
• Tagging scheme: BIES (Begin/Inside/End/Single)

License

Schift License v2.0 — Apache 2.0 base with a revenue threshold. Free for everyone under $10M annual revenue. Research, education, and non-profit use always permitted. Companies above the threshold: contact hello@schift.io.

Citation

@software{schift_ko_pii_2026,
  author = {Schift Inc.},
  title = {schift-ko-pii: Korean PII Detection Model},
  year = {2026},
  url = {https://huggingface.co/schift-io/schift-ko-pii-v4},
}