pyrule

English | 中文

保序且区分类型的列表去重。

安装

pip install pyrule

用法

from pyrule import dedupe, dedupe_key

dedupe(["b", "a", "b"])                  # ['b', 'a']
dedupe([1, 1.0, True])                   # [1, 1.0, True]
dedupe([{"a": 1}, {"a": 1}, {"a": 2}])   # two dicts kept
dedupe_key(1), dedupe_key(1.0)           # ('int', '1'), ('float', '1.0')

说明

标量按 (类型名, 值) 判重，所以 1、1.0、True 保持三条——从 CSV / JSON / 数据库出来的就是这样；其余类型按规范化 JSON 判重，字段顺序不同视为同一条。

许可证

MIT

English

Order-preserving, type-aware deduplication for mixed lists.

Install

pip install pyrule

Usage

from pyrule import dedupe, dedupe_key

dedupe(["b", "a", "b"])                  # ['b', 'a']
dedupe([1, 1.0, True])                   # [1, 1.0, True]
dedupe([{"a": 1}, {"a": 1}, {"a": 2}])   # two dicts kept
dedupe_key(1), dedupe_key(1.0)           # ('int', '1'), ('float', '1.0')

Notes

Scalars are keyed by (type name, repr) so 1, 1.0 and True stay distinct — that is how they arrive from CSV, JSON and SQL. Other values are keyed by canonical JSON, so key order does not matter.

License

MIT