Evaluation of Generative AI Models
Project description
GENAI EVALUATION
GenAI Evaluation is a library which contains methods to evaluate differences in Real & Synthetic Data.
Authors
Installation
The package can be installed with
pip install genai_evaluation
Tests
The test can be run by cloning the repo and running:
pytest tests
If this does not work, the package might not currently be findable. In that case, please install it locally with:
pip install -e .
Usage
Start by importing the class
from genai_evaluation import multivariate_ecdf, ks_distance
Assuming we have two pandas dataframes (Real & Synthetic) and only numerical columns, we pass them to the multivariate_ecdf function which returns the computed multivariate ECDFs of both.
ecdf_real, ecdf_synth = multivariate_ecdf(real_data, synthetic_data, n_nodes = 1000, verbose = True)
We then calculate the multivariate KS Distance between the ECDFs
ks_stat = ks_distance(ecdf_real, ecdf_synth)
Motivation
The motivation for this package comes from Dr. Vincent Granville's paper Generative AI Technology Break-through: Spectacular Performance of New Synthesizer
If you have any tips or suggestions, please contact us on email.
0.1.0 (2023-09-11)
- First release on PyPI.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for genai_evaluation-0.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9718d1d46c422d2336b6139f857b4492e15e341f4e02257d52160fc8dea20482 |
|
MD5 | 54a3774910e9ba1a6f5e5f178941e12d |
|
BLAKE2b-256 | 0e44aa7bb85ac6db1a99efb54955d7d233b270ea175559da59da64292c9a86b1 |