londonaicentre-mesa-local 1.3.1

Serve MESA models locally

MESA local

Serve MESA models locally.

• ⬇️ Downloads weights from S3

• 📦 Unpacks

• 🚀 Serves via a local OpenAI-compatible server

Prerequisites

Software

• Python 3.12

Hardware

• A GPU with >=24GB VRAM (tested on NVIDIA A30)

Configuration

• Create a file called .env in the directory where you intend to run this package. Populate it with the details you have been provided with in the following format:

MODEL_NAME=
WEIGHTS_ID=
WEIGHTS_KEY=

Installation

1. (Recommended) Create a virtual environment and activate it:

   python -m venv .venv
   source .venv/bin/activate
   

2. Install this package: pip install londonaicentre-mesa-local.

Usage

CLI

1. Note command line arguments:

   Argument	Description
   -v, --verbose	Enable debug output (optional)

2. Start the server as follows: mesalocal [args].

Clients

OpenAI (example with Oncollama)

1. Interact with the server using the OpenAI client in python:

   from openai import OpenAI
   from oncoschema.prompt_builder import PromptBuilder # pip install londonaicentre-oncoschema
   
   client = OpenAI(
       base_url="http://localhost:5000/v1",
       api_key="blank" 
   )
   
   response = client.chat.completions.create(
       model="oncollama3betav01",
       messages=[
           {"role": "system", "content": PromptBuilder().build_main_prompt()},
           {"role": "user", "content": "Diagnosis 01/01/26..."}
       ]
   )
   
   print(response.choices[0].message.content)
   

License

This project uses a proprietary license (see LICENSE).