plato-tile-dedup 0.1.0

4-stage tile similarity — exact, keyword Jaccard, embedding cosine, structure

plato-tile-dedup

4-stage tile similarity — exact, keyword Jaccard, embedding cosine, structure.

Part of the PLATO framework — deterministic AI knowledge management through tile-based architecture.

Installation

pip install plato-tile-dedup

Usage

from plato_tile_dedup import TileDeduplicator

dedup = TileDeduplicator(threshold=0.85)

tiles = [
    {"id": "1", "content": "Pythagorean triples snap to exact coordinates"},
    {"id": "2", "content": "Pythagorean triples snap to exact coordinates"},  # duplicate
    {"id": "3", "content": "Constraint theory maps continuous vectors to discrete space"},
]

unique = dedup.dedup_batch(tiles)
print(len(unique))  # 2 (tile 2 removed as duplicate of tile 1)

4 Similarity Stages

Stage	Weight	Method
Exact	0.1	Content hash comparison
Keyword	0.3	Token-level Jaccard overlap
Embedding	0.5	Cosine similarity (requires embedding model)
Structure	0.1	Domain and question type matching

Zero external dependencies. Compatible with Python 3.8+.

GitHub