Skip to main content

A Python library for Automated Exploratory Data Analysis, Automated Data Cleaning and Automated Data Preprocessing For Machine Learning and Natural Language Processing in Python.

Project description

Data-Purifier

A Python library for Automated Exploratory Data Analysis, Automated Data Cleaning and Automated Data Preprocessing For Machine Learning and Natural Language Processing Applications in Python.

PyPI version License Python Version PyPi Downloads

Features

  • It gives shape, number of categorical and numerical features, description of the dataset, and also the information about the number of null values and their respective percentage.

  • For understanding the distribution of datasets and getting useful insights, there are many interactive plots generated where the user can select his desired column and the system will automatically plot it. Plot includes

    1. Count plot
    2. Correlation plot
    3. Joint plot
    4. Pair plot
    5. Pie plot

Get Started

Install the packages

pip install data-purifier
python -m spacy download en_core_web_sm

Load the module

from datapurifier import Mleda, Nleda, Nlpurifier

Load the dataset and let the magic of automated EDA begin

df = pd.read_csv("./datasets/iris.csv")
ae = Mleda(df)
ae

For Automated EDA and Automated Data Cleaning of NL dataset, load the dataset and pass the dataframe along with the targeted column containing textual data.

nlp_df = pd.read_csv("./datasets/twitter16m.csv", header=None, encoding='latin-1')
nlp_df.columns = ["tweets","sentiment"]

Automated EDA

For Basic EDA, pass the argument basic as argument in constructor

%%time
eda = Nlpeda(nlp_df, "tweets", analyse="basic")
eda.df

For Word based EDA, pass the argument word as argument in constructor

%%time
eda = Nlpeda(nlp_df, "tweets", analyse="word")
eda.unigram_df # for seeing unigram datfarame

Automated Data Cleaning

pure = Nlpurifier(nlp_df, "tweets")

View the processed and purified dataframe

pure.df

Example: https://colab.research.google.com/drive/1J932G1uzqxUHCMwk2gtbuMQohYZsze8U?usp=sharing

Python Package: https://pypi.org/project/data-purifier/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

data_purifier-0.1.9-py3-none-any.whl (20.6 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page