An API to measure evaluation criteria (ex: faithfulness) of generative AI outputs
Project description
LastMile AI Eval
Library of tools to evaluate your RAG system.
Setup
- Get a LastMile API token (see section below)
- Install this library:
pip install lastmile-eval
- Gather your data that needs evaluation
- See the
examples/
folder for API usage
LastMile API token
To get a LastMile AI token, please go to the LastMile token's webpage. You can create an account with Google or Github and then click the "Create new token" in the "API Tokens" section. Once a token is created, be sure to save it somewhere since you won't be able to see the value of it from the website again (though you can create a new one if that happens).
Please be careful not to share your token on GitHub. Instead we recommend saving it under your project’s (or home directory) .env
file as: LASTMILE_API_TOKEN=<TOKEN_HERE>
, and use loadenv instead. See the examples/
folder for how to do this.
LLM Provider Tokens (.env
file)
In order to use LLM-based evaluators, add your other API tokens to your .env file.
Example: OPENAI_API_KEY=<TOKEN_HERE>
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for lastmile_eval-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 54dbf4814039607a5994c5f81c148c9e0520ad729b32b39729bffc6a20883a64 |
|
MD5 | 1c8be011e34e501f04ea79986a4b87a6 |
|
BLAKE2b-256 | e5941f3cdae03d90444bbad7cf2d4c6815c7612a10e1437e39cc5f8c1483454c |