dirty-cat 0.2.1

Machine learning with dirty categories.

dirty_cat is a Python module for machine-learning on dirty categorical variables.

Website: https://dirty-cat.github.io/

dirty_cat’s SuperVectorizer automatically turns pandas data frames into numerical arrays suitable for learning.

For a detailed description of the problem of encoding dirty categorical data, see Similarity encoding for learning with dirty categorical variables [1] and Encoding high-cardinality string categorical variables [2].

Installation

Dependencies

dirty_cat requires:

• Python (>= 3.6)
• NumPy (>= 1.16)
• SciPy (>= 1.2)
• scikit-learn (>= 0.21.0)
• pandas (>= 1.1.5)

Optional dependency:

• python-Levenshtein for faster edit distances (not used for the n-gram distance)

User installation

If you already have a working installation of NumPy and SciPy, the easiest way to install dirty_cat is using pip

pip install -U --user dirty_cat

Other implementations

• Spark ML: https://github.com/rakutentech/spark-dirty-cat

References

[1]	Patricio Cerda, Gaël Varoquaux, Balázs Kégl. Similarity encoding for learning with dirty categorical variables. 2018. Machine Learning journal, Springer.

[2]	Patricio Cerda, Gaël Varoquaux. Encoding high-cardinality string categorical variables. 2020. IEEE Transactions on Knowledge & Data Engineering.