language: ko license: other license_name: schift-open-1.0 license_link: LICENSE library_name: onnxruntime pipeline_tag: token-classification tags:

• pii
• korean
• ner
• privacy
• onnx
• token-classification datasets:
• custom model-index:
• name: schift-ko-pii-v4 results:
  • task: type: token-classification name: Korean PII Detection metrics:
    • type: f1 value: 0.863 name: F1 (raw, no postprocess)
    • type: f1 value: 0.968 name: F1 (with postprocess)

schift-ko-pii-v4

111M parameter Korean PII detector that outperforms models 3x its size.

Fine-tuned from klue/roberta-base with LoRA on Korean legal, financial, and administrative texts. Exported to ONNX INT8 for CPU-only deployment (107 MB).

Benchmark: schift-ko-pii-v4 vs LFM2.5-Encoder-350M-PII-Detector

Same benchmark, same scoring (50% overlap match), no postprocessing on either side.

With postprocessing enabled (postprocess=True):

Category breakdown (raw model, no postprocess)

LFM2.5 is a strong multilingual model, but it was not trained on Korean-specific formats: vehicle plates (52가1234), resident registration numbers (850205-1234567), Korean date expressions (2024년 3월 15일), or rare Korean surnames (황보, 남궁, 독고).

Quick start

pip install schift-ko-pii

from schift_ko_pii import detect

# Raw model output
spans = detect("피고 김민수의 전화번호는 010-1234-5678이다.")
# [
#   {"start": 3, "end": 6, "label": "private_person", "score": 0.99, "text": "김민수"},
#   {"start": 14, "end": 27, "label": "private_phone", "score": 0.98, "text": "010-1234-5678"},
# ]

# With postprocessing (regex validation + structured-ID rules, higher precision)
spans = detect("주민번호 850205-1234567을 확인.", postprocess=True)

Postprocessing

The postprocess=True flag applies Korean-specific rules:

• Regex validation for structured IDs (resident numbers, passports, vehicle plates, bank accounts)
• Luhn/checksum verification where applicable
• Context-aware span merging
• False-positive suppression for legal case numbers and statute references

This raises strong-category F1 from 0.893 to 0.968.

API (free)

For production use without managing model files:

from schift import Schift

client = Schift(api_key="...")  # free at schift.io
result = client.pii.redact("김민수의 전화번호는 010-1234-5678입니다.")
# Postprocessing is always enabled on the API.

Labels

Benchmark

The benchmark suite is included in this repository under benchmark/.

# Benchmark this model
python benchmark/run_benchmark.py

# With postprocess
python benchmark/run_benchmark.py --postprocess

# Compare any HuggingFace model
python benchmark/run_benchmark.py --hf-model LiquidAI/LFM2.5-Encoder-350M-PII-Detector

93 hand-crafted cases covering Korean legal, investigative, and administrative text. Categories: person (standard/rare/short/foreign/hanja), phone, email, address (urban/rural), dates, URLs, resident IDs, business IDs, bank accounts, passports, vehicle plates, IP addresses, plus negative cases (legal case numbers, statute references, Hanja legal terms).

Model details

• Base model: klue/roberta-base (111M params)
• Training: LoRA adapter on ~20k Korean legal/financial/admin examples
• Export: ONNX FP32 → INT8 dynamic quantization
• Inference: ONNX Runtime CPU, single thread, 107 MB
• Max length: 512 tokens
• Tagging scheme: BIES (Begin/Inside/End/Single)

License

Schift Open License v1.0 — Apache 2.0 base with a revenue threshold. Free for everyone under $10M annual revenue. Research, education, and non-profit use always permitted. Companies above the threshold: contact licensing@schift.io.

Citation

@software{schift_ko_pii_2026,
  author = {Schift Inc.},
  title = {schift-ko-pii: Korean PII Detection Model},
  year = {2026},
  url = {https://huggingface.co/schift-io/schift-ko-pii-v4},
}