LLM Labeling UI is an open source project for large language model data labeling
Project description
LLM Labeling UI
About
WARNING: This software is mainly developed according to my personal habits and is still under development. We are not responsible for any data loss that may occur during your use.
LLM Labeling UI is a project fork from Chatbot UI, and made the following modifications to make it more suitable for large language model data labeling tasks.
- The backend code is implemented in python, the frontend code is precompiled, so it can run without a nodejs environment
- The Chatbot UI uses localStorage to save data, with a size limit of 5MB, the LLM Labeling UI can load local data when starting the service, with no size limit
- Web interaction:
- You can view data in pages
- You can directly modify/delete model's response results
- A confirmation button has been added before deleting the conversation message
- Display the number of messages and token length in the current dialogue
- You can modify the system prompt during the dialogue
Quick Start
pip install llm-labeling-ui
1. Provide OpenAI API Key
You can provide openai api key before start server or configure it later in the web page.
export OPENAI_API_KEY=YOUR_KEY
export OPENAI_ORGANIZATION=YOUR_ORG
2. Start Server
llm-labeling-ui start --data chatbot-ui-v4-format-history.json --tokenizer meta-llama/Llama-2-7b
--data
: Chatbot-UI-v4 format, here is an example. Before the service starts, achatbot-ui-v4-format-history.sqlite
file will be created based onchatbot-ui-v4-format-history.json
. All your modifications on the page will be saved into the sqlite file. If thechatbot-ui-v4-format-history.sqlite
file already exists, it will be automatically read.--tokenizer
is used to display how many tokens the current conversation on the webpage contains. Please note that this is not the token consumed by calling the openai api.
3. Export data from sqlite
llm-labeling-ui export --db-path chatbot-ui-v4-format-history.sqlite
By default exported data will be generated in the same directory as --db-path
, and the file name will be added with a timestam.
Other features
By default, all command will not perform operations on the database, it will only print some info to preview. Adding the --run
can execute the command.
- Remove conversation which is prefix of another conversation
llm-labeling-ui remove-prefix-conversation --db-path chatbot-ui-v4-format-history.sqlite
- Delete string from conversation
llm-labeling-ui delete-string --db-path chatbot-ui-v4-format-history.sqlite --string "some text"
- Remove duplicate conversations, only keep one
llm-labeling-ui remove-duplicate-conversation --db-path chatbot-ui-v4-format-history.sqlite
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llm_labeling_ui-0.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | acb4194f5d2a9559d1845ff01991302d48bc37d5da2f3a884537cf0f85af97b8 |
|
MD5 | 3065dcf668d45abd8424adf424498bd4 |
|
BLAKE2b-256 | 4be5b2fe8706da4851a6e784054568a31e062164482f288fa4c58a877d02ff6c |