Skip to main content

Evaluation of Generative AI Models

Project description

GENAI EVALUATION

PyPI version Documentation

GenAI Evaluation is a library which contains methods to evaluate differences in Real and Synthetic Data.

Functions

  • multivariate_ecdf: Computes joint or multivariate ECDF in contrast to the univariate capabilities provided by packages like statsmodels
  • ks_statistic: Calculates the KS Statistic for two multivariate ECDFs

Authors

Installation

The package can be installed with

pip install genai_evaluation

Tests

The test can be run by cloning the repo and running:

pytest tests

In case of any issues running the tests, please run them after installing the package locally:

pip install -e .

Usage

Start by importing the class

from genai_evaluation import multivariate_ecdf, ks_statistic

Assuming we have two pandas dataframes (Real & Synthetic) and only numerical columns, we pass them to the multivariate_ecdf function which returns the computed multivariate ECDFs of both.

query_str, ecdf_real, ecdf_synth = multivariate_ecdf(real_data, synthetic_data, n_nodes = 1000, verbose = True)

We then calculate the multivariate KS Distance between the ECDFs

ks_stat = ks_statistic(ecdf_real, ecdf_synth)

Motivation

The motivation for this package comes from Dr. Vincent Granville's paper Generative AI Technology Break-through: Spectacular Performance of New Synthesizer

If you have any tips or suggestions, please contact us on email.

History

0.1.0 (2023-09-11)

  • First release on PyPI.

0.1.1 (2023-09-11)

Corrected

  • Function name from compute_ecdf to multivariate_ecdf

0.1.2 (2023-09-11)

Enhanced

  • Added a new parameter verbose in multivariate ECDF function

0.1.3 (2023-09-11)

Corrected

  • Removed unecessary docstrings from code

0.1.4 (2023-09-11)

Fixed

  • Resolved issues with special characters in the column names

0.1.5 (2023-09-11)

Fixed

  • Earlier version considered underscore as a special character. That is rectified in this version

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

genai_evaluation-0.1.5.tar.gz (8.0 kB view details)

Uploaded Source

Built Distribution

genai_evaluation-0.1.5-py2.py3-none-any.whl (6.0 kB view details)

Uploaded Python 2Python 3

File details

Details for the file genai_evaluation-0.1.5.tar.gz.

File metadata

  • Download URL: genai_evaluation-0.1.5.tar.gz
  • Upload date:
  • Size: 8.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.4

File hashes

Hashes for genai_evaluation-0.1.5.tar.gz
Algorithm Hash digest
SHA256 b23f7f2b118a7a5f6f6fbe3791574a44533e23bede66089cc91e4d1ed2f7bcb9
MD5 756a51b61c3f16475f1d63fb8b3a9e45
BLAKE2b-256 d5bb04c2269abf2fcde09e74ba597a621c9bc26c9c44200ef8899d9947cd4876

See more details on using hashes here.

File details

Details for the file genai_evaluation-0.1.5-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for genai_evaluation-0.1.5-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 2c3c2f6a53d73e1fbb32a7937b412676726e092d9563a075085d5e03e29d755d
MD5 dabd76718f4c9267ae7f335fee2ec62a
BLAKE2b-256 37cc66fc6a057d77ab19120e64f63d99e8841fea635c34d38b60239580d29359

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page