POLARS-DEUNICODE-STRING

This is a simple polars-plugin that de-unicodes a string using the deunicode crate.

Installation

pip install polars-deunicode-string

Basic Example

import polars as pl
import polars_deunicode_string as deunicode


df = pl.DataFrame(
    {
        "name": ["Jhon", "María", "José", "Eva-lin"],
        "city": ["Bogotá", "Tunja", "Medellín", "Nariño"],
    }
)
normalized_df = df.select([
    deunicode.decode_string(pl.col(pl.String))
])
print(df)
>>> Data:

shape: (4, 2)
┌─────────┬──────────┐
│ name    ┆ city     │
│ ---     ┆ ---      │
│ str     ┆ str      │
╞═════════╪══════════╡
│ Jhon    ┆ Bogotá   │
│ María   ┆ Tunja    │
│ José    ┆ Medellín │
│ Eva-lin ┆ Nariño   │
└─────────┴──────────┘

print(normalized_df)
>>> Normalized Data:

shape: (4, 2)
┌─────────┬──────────┐
│ name    ┆ city     │
│ ---     ┆ ---      │
│ str     ┆ str      │
╞═════════╪══════════╡
│ Jhon    ┆ Bogota   │
│ Maria   ┆ Tunja    │
│ Jose    ┆ Medellin │
│ Eva-lin ┆ Narino   │
└─────────┴──────────┘