Skip to main content

Information gain utilities

Project description


Implementation of information gain algorithm. There seems to be a debate about how the information gain metric is defined. Whether to use the Kullback-Leibler divergence or the Mutual information as an algorithm to define information gain. This implementation uses the information gain calculation as defined below:

Information gain definitions

Information gain calculation

Definition from information gain calculation (retrieved 2018-07-13). Let Attr be the set of all attributes and Ex the set of all training examples, value(x, a) with x in Ex defines the value of a specific example x for attribute a in Attr, H specifies the entropy. The values(a) function denotes the set of all possible values of attribute a in Attr. The information gain for an attribute a in Attr is defined as follows:

Information gain formula

Intrinsic value calculation

Definition from information gain calculation (retrieved 2018-07-13).

Intrinsic value calculation

Information gain ratio calculation

Definition from information gain calculation (retrieved 2018-07-13).

Intrinsic value calculation


To install the package via pip use:

pip install info_gain

To clone the package from the git repository use:

git clone


Import the info_gain module with:

from info_gain import info_gain

The imported module has supports three methods:

  • info_gain.info_gain(Ex, a) to compute the information gain.
  • info_gain.intrinsic_value(Ex, a) to compute the intrinsic value.
  • info_gain.info_gain_ratio(Ex, a) to compute the information gain ratio.


from info_gain import info_gain

# Example of color to indicate whether something is fruit or vegatable
produce = ['apple', 'apple', 'apple', 'strawberry', 'eggplant']
fruit   = [ True  ,  True  ,  True  ,  True       ,  False    ]
colour  = ['green', 'green', 'red'  , 'red'       , 'purple'  ]

ig  = info_gain.info_gain(fruit, colour)
iv  = info_gain.intrinsic_value(fruit, colour)
igr = info_gain.info_gain_ratio(fruit, colour)

print(ig, iv, igr)

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for info-gain, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size info_gain-1.0.1-py3-none-any.whl (3.3 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size info_gain-1.0.1.tar.gz (3.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page