Skip to main content

Python library for the implementations of general and weighted naive Bayes (WNB) classifiers.

Project description

wnb logo
General and weighted naive Bayes classifiers
Scikit-learn-compatible

Lastest Release PyPI Version Python Versions
GitHub Workflow Status (build) PyPI License PyPi Downloads

Introduction

Naive Bayes is often recognized as one of the most popular classification algorithms in the machine learning community. This package takes naive Bayes to a higher level by providing its implementations in more general and weighted settings.

General naive Bayes

The issue with the well-known implementations of the naive Bayes algorithm (such as the ones in sklearn.naive_bayes module) is that they assume a single distribution for the likelihoods of all features. Such an implementation can limit those who need to develop naive Bayes models with different distributions for feature likelihood. And enters WNB library! It allows you to customize your naive Bayes model by specifying the likelihood distribution of each feature separately. You can choose from a range of continuous and discrete probability distributions to design your classifier.

Weighted naive Bayes

Although naive Bayes has many advantages such as simplicity and interpretability, its conditional independence assumption rarely holds true in real-world applications. In order to alleviate its conditional independence assumption, many attribute weighting naive Bayes (WNB) approaches have been proposed. Most of the proposed methods involve computationally demanding optimization problems that do not allow for controlling the model's bias due to class imbalance. Minimum Log-likelihood Difference WNB (MLD-WNB) is a novel weighting approach that optimizes the weights according to the Bayes optimal decision rule and includes hyperparameters for controlling the model's bias. WNB library provides an efficient implementation of gaussian MLD-WNB.

Installation

The easiest way to install the wnb library is by using pip:

pip install wnb

This library is shipped as an all-in-one module implementation with minimalistic dependencies and requirements. Furthermore, it fully adheres to Scikit-learn API ❤️.

Getting started ⚡️

Here, we show how you can use the library to train general and weighted naive Bayes classifiers.

General naive Bayes

A general naive Bayes model can be set up and used in four simple steps:

  1. Import the GeneralNB class as well as Distribution enum class
from wnb import GeneralNB, Distribution as D
  1. Initialize a classifier and specify the likelihood distributions
gnb = GeneralNB(distributions=[D.NORMAL, D.CATEGORICAL, D.EXPONENTIAL])
  1. Fit the classifier to a training set (with three features)
gnb.fit(X, y)
  1. Predict on test data
gnb.predict(X_test)

Weighted naive Bayes

An MLD-WNB model can be set up and used in four simple steps:

  1. Import the GaussianWNB class
from wnb import GaussianWNB
  1. Initialize a classifier
wnb = GaussianWNB(max_iter=25, step_size=1e-2, penalty="l2")
  1. Fit the classifier to a training set
wnb.fit(X, y)
  1. Predict on test data
wnb.predict(x_test)

Compatibility with Scikit-learn

The wnb library fully adheres to the Scikit-learn API, ensuring seamless integration with other Scikit-learn components and workflows. This means that users familiar with Scikit-learn will find the WNB classifiers intuitive to use.

Both Scikit-learn classifiers and WNB classifiers share these well-known methods:

  • fit(X, y)
  • predict(X)
  • predict_proba(X)
  • predict_log_proba(X)
  • score(X, y)
  • get_params()
  • set_params(**params)
  • etc.

By maintaining this consistency, WNB classifiers can be easily incorporated into existing machine learning pipelines and processes.

Benchmarks

We conducted benchmarks on three datasets, Breast Cancer, Digits, and Wine, to evaluate the performance of WNB classifiers and compare them with their Scikit-learn counterpart, GaussianNB. The results show that WNB classifiers generally perform better in certain cases.

Dataset Scikit-learn Classifier Accuracy WNB Classifier Accuracy
Breast Cancer GaussianNB 0.939 GaussianWNB 0.951
Digits GaussianNB 0.838 GeneralNB 0.889
Wine GaussianNB 0.974 GeneralNB 0.981

These benchmarks highlight the potential of WNB classifiers to provide better performance in certain scenarios by allowing more flexibility in the choice of distributions and incorporating weighting strategies.

The benchmark scripts used to obtain these results can be found under tests/benchmarks/ directory.

Tests

To run the tests, make sure to clone the repository and install the development requirements:

pip install -r requirements_dev.txt

Then, run pytest:

pytest

Support us 🤝

You can support the project in the following ways:

⭐ Star WNB on GitHub (click the star button in the top right corner)

💡 Provide your feedback or propose ideas in the Issues section

📰 Post about WNB on LinkedIn or other platforms

Citation 📚

If you utilize this repository, please consider citing it with:

@misc{wnb,
  author = {Mohammd Mehdi Samsami},
  title = {WNB: General and weighted naive Bayes classifiers},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/msamsami/weighted-naive-bayes}},
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

wnb-0.2.5.tar.gz (15.3 kB view details)

Uploaded Source

Built Distribution

wnb-0.2.5-py3-none-any.whl (18.8 kB view details)

Uploaded Python 3

File details

Details for the file wnb-0.2.5.tar.gz.

File metadata

  • Download URL: wnb-0.2.5.tar.gz
  • Upload date:
  • Size: 15.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.9.19

File hashes

Hashes for wnb-0.2.5.tar.gz
Algorithm Hash digest
SHA256 ec995ca87035180e37f395198079badb556c315faab0a38040e4b79dfdf541a0
MD5 4ae3a0aadab33409ed1a59f533ea1470
BLAKE2b-256 c400e36fad68c3ab2328beaf1b5fe7a14c440fcf183aacf2929e80848d75de60

See more details on using hashes here.

File details

Details for the file wnb-0.2.5-py3-none-any.whl.

File metadata

  • Download URL: wnb-0.2.5-py3-none-any.whl
  • Upload date:
  • Size: 18.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.9.19

File hashes

Hashes for wnb-0.2.5-py3-none-any.whl
Algorithm Hash digest
SHA256 7c7e73b75fb62c561965f64b12830901beb5ca5aa1620d1a44d2572984fc4cb3
MD5 81b35d850fcf5817e2225a49ce6d98cb
BLAKE2b-256 bca750a47385f4ab8bffe4bff2ab00daa446b010730c471f63ccaac35ce2e9a2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page