LLMTag

A simple interface to label clinical data using local large language models (LLMs)

Features

1. Base code of getting started with LLMs
2. Try LLM's without PHI/PII issues and GPUs
3. Simple interface for several tasks
   • Label Documents
   • Fine-tune (upcoming)
   • RAG (upcoming)
4. Be in control
   1. Try any of the latest models
   2. Control Context Length
   3. Tailor to your specific need

Getting Started (Contributors)

1. Clone the llmtag repo
2. Download weights - any llama2 compatible model should work
   • Get llama-7B-chat weights from HF
   • Save it under ./models/7B
3. Add the model path to the environment variable MODEL
   • create .env file in root directory of repository (e.g. touch .env)
   • copy and paste below or define your own path to the model binary (actual model weights)
   • .env

     MODEL=./models/7B/llama-2-7b-chat.Q4_K_M.gguf
     

4. Initialize the environment with poetry install (if new to poetry, please check this)
5. For leveraging GPUs, please check - llama-cpp-python
   • Follow the instructions for installation as per your machine specifications
   • For simple CPU use case, can resort to not using GPU, but will be very time intensive, orders of magnitude more
   • Ignore this step for now unless you know better

Post installation

1. Run all tests using poetry run python -m pytest tests/
2. Run the default example: poetry run python -m llmtag

Results

Raw Clinical notes

LLM labeled notes

Metrics on simulated data

Confusion Matrix

n = 50

Libraries Used

1. llama.cpp
2. llama-cpp-python
3. Model Weights
4. poetry