language: ko license: other license_name: schift-2.0 license_link: LICENSE library_name: transformers pipeline_tag: token-classification tags:

• pii
• korean
• ner
• privacy
• token-classification datasets:
• custom

schift-ko-pii-v6

34.1M parameter Korean PII detector built on a dual-path LoRA encoder.

The 0.3.3 package loads the schift-io/schift-ko-pii-v6 checkpoint and adds Korean structured-PII postprocessing for identifiers, contact details, addresses, URLs, and secrets.

Release status

This PyPI release is schift-ko-pii 0.3.3 and is the v6 package lane. Postprocessing is enabled by default so structured PII categories are included when users call detect(text).

The Cloud Run ONNX service under services/pii is a separate deployment lane; this package does not bundle its ONNX artifacts.

Quick start

pip install schift-ko-pii

from schift_ko_pii import detect

spans = detect("피고 김민수의 전화번호는 010-1234-5678이다.")
# [
#   {"start": 3, "end": 6, "label": "private_person", "score": <model score>, "text": "김민수"},
#   {"start": 14, "end": 27, "label": "private_phone", "score": 0.99, "text": "010-1234-5678"},
# ]

# Encoder heads only (person / address / organization), no structured IDs
spans = detect("주민번호 850205-1234567을 확인.", postprocess=False)

Postprocessing

Postprocessing is on by default. Phone numbers, emails, resident registration numbers and other structured IDs come from these rules, not from the encoder heads, so postprocess=False drops those categories entirely.

It applies Korean-specific rules:

• Regex validation for structured IDs (resident numbers, passports, vehicle plates, bank accounts)
• Luhn/checksum verification where applicable
• Context-aware span merging
• False-positive suppression for legal case numbers and statute references

API (free)

For production use without managing model files:

from schift import Schift

client = Schift(api_key="...")  # free at schift.io
result = client.pii.redact("김민수의 전화번호는 010-1234-5678입니다.")
# Postprocessing is always enabled on the API.

Labels

Benchmark

The benchmark suite is included in this repository under benchmark/.

# Benchmark this model
python benchmark/run_benchmark.py

# With postprocess
python benchmark/run_benchmark.py --postprocess

# Compare any HuggingFace model
python benchmark/run_benchmark.py --hf-model LiquidAI/LFM2.5-Encoder-350M-PII-Detector

93 hand-crafted cases covering Korean legal, investigative, and administrative text. Categories: person (standard/rare/short/foreign/hanja), phone, email, address (urban/rural), dates, URLs, resident IDs, business IDs, bank accounts, passports, vehicle plates, IP addresses, plus negative cases (legal case numbers, statute references, Hanja legal terms).

Model details

• Checkpoint: schift-io/schift-ko-pii-v6
• Architecture: dual-path LoRA encoder with person, address, and organization heads
• Training: LoRA adapter on Korean legal/financial/admin examples
• Format: safetensors release source
• Inference: custom transformers/PyTorch dual-path loader
• Max length: 512 tokens
• Tagging scheme: O/B/I/E/S

License

Schift License v2.0 — Apache 2.0 base with a revenue threshold. Free for everyone under $10M annual revenue. Research, education, and non-profit use always permitted. Companies above the threshold: contact hello@schift.io.

Citation

@software{schift_ko_pii_2026,
  author = {Schift Inc.},
  title = {schift-ko-pii: Korean PII Detection Model},
  year = {2026},
  url = {https://huggingface.co/schift-io/schift-ko-pii-v6},
}