Introduction
Liken is a library providing enhanced deduplication tooling for DataFrames.
The key features are:
- Near deduplication tooling
- Exploratory duplicate-rate profiling
- Fuzzy string matching deduper
- TF-IDF tokenization deduper
- LSH tokenization deduper
- Jaccard set deduper
- Cosine set deduper
- Pandas API extension
- Composable, rules-based, deduplication pipelines
- Predicate dedupers for rules
- Record linkage and canonicalization
- Built-in Preprocessors
- Pandas, Polars, Modin, Ray, Dask and PySpark support
- Customizable in pure Python
- Synthetic record creation
- Easy to understand syntax
- Dummy datasets for practice
A flexible API
Checkout the API Documentation
Installation
uv pip install liken
Example
import liken as lk
df = lk.dedupe(df).apply(lk.fuzzy()).drop_duplicates("address")
License
This project is licensed under the Apache-2.0 License. See the LICENSE file for more details.
1 maintainer
Metadata
1 maintainer
Release files for liken 0.11.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| liken-0.11.0.tar.gz | 45.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| liken-0.11.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 121.0 kB
Release files / liken-0.11.0.tar.gz
| Download URL | liken-0.11.0.tar.gz |
|---|---|
| Size | 45.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5a0120970a9dc45e9f27239ac43757ed4f921ac40104bcbc2fa922f9889baeb1
|
|
BLAKE2b-256 checksum How to use checksums |
4d7a7f7b96f9546270cb4ff191c5a0e865fdc33ecc5afdc4e145ae155f5d8f97
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / liken-0.11.0-py3-none-any.whl
| Download URL | liken-0.11.0-py3-none-any.whl |
|---|---|
| Size | 75.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b438db5a08400ea082f6e5fe223afe15836d2f9d114683db7d4eac72d88d1075
|
|
BLAKE2b-256 checksum How to use checksums |
f86a2d4bc04f658fdd415cac736f7c2830a3ecd3eb1c09e56a8263db30b6cb25
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
