Trustwise PyPi Package
Project description
🦉 Trustwise Safety Module
The Trustwise Safety Module is a Python package designed to evaluate the RAG pipelines of a Language Model (LLM). It provides an aggregated Safety Score, along with various other individual scores and logs events during the query and index construction of the pipelines. Users can conveniently check event logs and safety score results stored in a MongoDB backend.
🔧 Installation
Install the Trustwise Safety Module using pip:
pip install trustwise
🔥 Features
- Evaluate RAG pipelines of an LLM.
- Aggregated Safety Score calculation.
- Event logging during query and index construction.
- Backend storage in MongoDB.
- Support for bulk scanning of responses from LLMs.
🚀 Usage with Llama Index
from llama_index.callbacks import CallbackManager
from trustwise.callback import TrustwiseCallbackHandler
from trustwise.request import request_eval
# Initialise Trustwise Callback Handler
user_id = "Enter your User ID"
scan_name = "Enter your Scan Name"
tw_callback = TrustwiseCallbackHandler(user_id=user_id, scan_name=scan_name)
# Configure the Handler with LlamaIndex Callback Manager
callback_manager = CallbackManager([tw_callback])
# Rest of the llamaindex code like indexing, llm response generation comes here
###### Evaluate LLM responses #######
scores = request_eval(user_id=user_id,scan_name=scan_name, query=query, response=response)
print(scores)
🔐 Trustwise API Key
Get your API Key by logging in through Github -> link
Contributing
We welcome contributions! If you find a bug, have a feature request, or want to contribute code, please create an issue or submit a pull request.
🪪 License
This project is licensed under the MIT License - see the LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for trustwise-3.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b8eab772405d7bac88e34ee8cb8009674143476300979939de0a58ac1287a75d |
|
MD5 | af92aeebc6d0d235db9590bef077fc7d |
|
BLAKE2b-256 | 4db6b8db9d79f6bf5e29b8f378662cbacdcf5a3bc8da912845244bdbf716c381 |