Skip to main content

A Python package for removing outliers in dataset using the Interquartile Range technique.

Project description

Outlier Removal using Interquartile Range - Python

Project 2 : UCS633

Submitted By: Khushnuma Grover 101703289


pypi: https://pypi.org/project/outlier-removal-101703289
git: https://github.com/khushgrover/outlier-removal-python.git


What are Outliers ?

There are certain things which, if are not done in the Exploratory Data Analysis phase, can affect further statistical/Machine Learning modelling. One of them is finding Outliers.

In statistics, an outlier is an observation point that is distant from other observations.

Interquartile Range Method

A robust method for labelling outliers is the IQR (interquartile range) method of outlier detection developed by John Tukey, the pioneer of exploratory data analysis. This was in the days of calculation and plotting by hand, so the datasets involved were typically small, and the emphasis was on understanding the story the data told. A box-and-whisker plot (also a Tukey contribution), shows this method in action.

A box-and-whisker plot uses quartiles (points that divide the data into four groups of equal size) to plot the shape of the data. The box represents the 1st and 3rd quartiles, which are equal to the 25th and 75th percentiles. The line inside the box represents the 2nd quartile, which is the median.

The interquartile range, which gives this method of outlier detection its name, is the range between the first and the third quartiles (the edges of the box). Tukey considered any data point that fell outside of either 1.5 times the IQR below the first – or 1.5 times the IQR above the third – quartile to be outside or far out. In a classic box-and-whisker plot, the 'whiskers' extend up to the last data point that is not outside.


How to use this package:

In this package this method is implemented. It take in the dataset csv file and outputs out a csv file in the rows having outlier values are removed. This package handles univariate datasets as well as multivariate. Each feature having outlier rows are removed.

OUTLIER-REMOVAL-KHUSHNUMA-101703289 can be run as in the following example:

Installing the package

On your Command Prompt run the following command:

pip install outlier-removal-101703289

Sample dataset

The dataset should be constructed with each row representing a data, and each column representing a criterion feature, ending with a target.


Feature Target
10 0.62
100 0.44
100 0.31
100 0.67
10 0.56

In Command Prompt:

>> remove-outlier data.csv

In Python IDLE:

>>> from outlier_removal.outlier import remove_outlier
>>> file1 = pd.read_csv('data.csv')
>>> remove_outlier(file1)

Output

Feature Target
100 0.44
100 0.31
100 0.67

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for outlier-removal-101703289, version 1.0.2
Filename, size File type Python version Upload date Hashes
Filename, size outlier_removal_101703289-1.0.2-py3-none-any.whl (4.7 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size outlier-removal-101703289-1.0.2.tar.gz (4.1 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page