Tools for testing, debugging, and evaluating LLM features.
Project description
Baserun
Baserun is the collaborative workspace for AI teams. Our mission is to simplify the testing, debugging, and evaluation of LLM features to help you get your app production-ready.
Quick Start
Install baserun
pip install baserun
Get your API key from the Baserun dashboard and set it as an environment variable:
export BASERUN_API_KEY="your_api_key_here"
Use our pytest plugin and start immediately logging to Baserun. By default all OpenAI completion and chat requests will be logged to Baserun. Logs are aggregated by test.
# test_module.py
import openai
def test_paris_trip():
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
temperature=0.7,
messages=[
{
"role": "user",
"content": "What are three activities to do in Paris?"
}
],
)
assert "Eiffel Tower" in response['choices'][0]['message']['content']
To run the test and log to baserun:
pytest --baserun test_module.py
...
========================Baserun========================
Test results available at: https://baserun.ai/runs/<id>
=======================================================
Custom logs
log
Logs a custom message to Baserun. If Baserun is not initialized, this function will have no effect.
Parameters
- message (str): The custom log message to be recorded.
- payload (Union[str, Dict]): The log's additional data, which can be either a string or a dictionary.
import baserun
def test_custom_log():
...
baserun.log("CustomEvent", payload={"key": "value"})
Production monitoring with baserun.trace
trace
Aggregates a trace of all logs generated by a function call to Baserun. By default all OpenAI completion and chat requests will be logged as part of the trace. If Baserun is not initialized, baserun.trace will have no effect.
Parameters
- function (Callable): The function to be traced.
- metadata (Optional[Dict]): Additional metadata to record alongside the traced logs.
Initialization
Add your BASERUN_API_KEY as an environment variable and then initialize Baserun at the entry point of your application.
import baserun
if env == 'production':
baserun.init()
Then either decorate the function that you would like to trace:
import baserun
@baserun.trace
def get_response():
...
or wrap the function and optionally pass any additional metadata:
import baserun
def get_response(message):
...
traced_get_response = baserun.trace(get_response, {"userId": 123})
traced_get_response(message)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.