Skip to main content

Graphical Hypergeometric Networks

Project description

Python PyPI Version License Github Forks GitHub Open Issues Project Status Downloads Downloads Sphinx arXiv Substack Open In Colab

HNET - Association ruled based networks using graphical Hypergeometric Networks.

Star this repo if you like it! ⭐️

Dashboard HNet!

HNet Dashboard

Summary

HNet stands for graphical Hypergeometric Networks, which is a method where associations across variables are tested for significance by statistical inference. The aim is to determine a network with significant associations that can shed light on the complex relationships across variables. Input datasets can range from generic dataframes to nested data structures with lists, missing values and enumerations.

Real-world data often contain measurements with both continuous and discrete values. Despite the availability of many libraries, data sets with mixed data types require intensive pre-processing steps, and it remains a challenge to describe the relationships between variables. The data understanding phase is crucial to the data-mining process, however, without making any assumptions on the data, the search space is super-exponential in the number of variables. A thorough data understanding phase is therefore not common practice.

Methods

We propose graphical hypergeometric networks (HNet), a method to test associations across variables for significance using statistical inference. The aim is to determine a network using only the significant associations in order to shed light on the complex relationships across variables. HNet processes raw unstructured data sets and outputs a network that consists of (partially) directed or undirected edges between the nodes (i.e., variables). To evaluate the accuracy of HNet, we used well known data sets and generated data sets with known ground truth. In addition, the performance of HNet is compared to Bayesian association learning.

Results

We demonstrate that HNet showed high accuracy and performance in the detection of node links. In the case of the Alarm data set we can demonstrate on average an MCC score of 0.33 + 0.0002 (P<1x10-6), whereas Bayesian association learning resulted in an average MCC score of 0.52 + 0.006 (P<1x10-11), and randomly assigning edges resulted in a MCC score of 0.004 + 0.0003 (P=0.49).

Conclusions

HNet overcomes processes raw unstructured data sets, it allows analysis of mixed data types, it easily scales up in number of variables, and allows detailed examination of the detected associations.

Documentation

Method overview

Installation

  • Install hnet from PyPI (recommended).
pip install -U hnet

Examples

  • Simple example for the Titanic data set
# Initialize hnet with default settings
from hnet import hnet
# Load example dataset
df = hnet.import_example('titanic')
# Print to screen
print(df)
#      PassengerId  Survived  Pclass  ...     Fare Cabin  Embarked
# 0              1         0       3  ...   7.2500   NaN         S
# 1              2         1       1  ...  71.2833   C85         C
# 2              3         1       3  ...   7.9250   NaN         S
# 3              4         1       1  ...  53.1000  C123         S
# 4              5         0       3  ...   8.0500   NaN         S
# ..           ...       ...     ...  ...      ...   ...       ...
# 886          887         0       2  ...  13.0000   NaN         S
# 887          888         1       1  ...  30.0000   B42         S
# 888          889         0       3  ...  23.4500   NaN         S
# 889          890         1       1  ...  30.0000  C148         C
# 890          891         0       3  ...   7.7500   NaN         Q

Play with the interactive Titanic results.

Example: Learn association learning on the titanic dataset

Example: Summarize results

Networks can become giant hairballs and heatmaps unreadable. You may want to see the general associations between the categories, instead of the label-associations. With the summarize functionality, the results will be summarized towards categories.

Example: Feature importance

Performance


Contribute

  • All kinds of contributions are welcome!

Citation

Please cite HNet in your publications if this is useful for your research. See column right for citation information.

Maintainer

  • Erdogan Taskesen, github: erdogant
  • Contributions are welcome.
  • If you wish to buy me a Coffee for this work, it is very appreciated :)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hnet-1.2.3.tar.gz (48.5 kB view details)

Uploaded Source

Built Distribution

hnet-1.2.3-py3-none-any.whl (48.9 kB view details)

Uploaded Python 3

File details

Details for the file hnet-1.2.3.tar.gz.

File metadata

  • Download URL: hnet-1.2.3.tar.gz
  • Upload date:
  • Size: 48.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for hnet-1.2.3.tar.gz
Algorithm Hash digest
SHA256 aab8d59a126dd2d394f28b324bdbfaa3ec08a665afaca63dfc5f94f1d4902e19
MD5 c8ecd0e3182751e95d6c0bc47a085e67
BLAKE2b-256 1ad1819f6b0117b9a555e823843a15cb63cd0c9c853ed5cda33e7937d8a2a839

See more details on using hashes here.

File details

Details for the file hnet-1.2.3-py3-none-any.whl.

File metadata

  • Download URL: hnet-1.2.3-py3-none-any.whl
  • Upload date:
  • Size: 48.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for hnet-1.2.3-py3-none-any.whl
Algorithm Hash digest
SHA256 8c19cc21dc52affa5d909642922fcb94af7e5adcfd338abaa54d57b8e641019d
MD5 dd5dcc80102135fe8c842bcc2bfdde1b
BLAKE2b-256 c835ee7366b7778c872b0d25f582051221ecbde9d20ee3914e8376bd48879d97

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page