Quantum LLM Client

quantum-llm-client is a Python client library designed to interact with OpenAI-compatible LLM server endpoints. It incorporates built-in prompt management using YAML configurations and features an automated financial advisor persona with response latency tracking.

Features

• OpenAI-Compatible Integration: Connect easily to any LLM server serving OpenAI-compatible endpoints.
• Prompt Management: Centralized YAML configuration (client_prompts.yml) for managing system instructions and user templates.
• Financial Advisor Persona: Pre-configured system prompts enforcing domain-specific guardrails.
• Latency Tracking: Automatically computes and returns the execution latency (in seconds) for each API call.
• Robust Error Handling: Gracefully catches API and unexpected exceptions without crashing execution.

Installation

Using pip

pip install quantum-llm-client

Using uv

uv pip install quantum-llm-client

Quickstart

from quantum_llm_client.qa_service import QAService

# Initialize the service with your endpoint's base URL
client = QAService(base_url="http://localhost:8000/v1")

# Query the client
query = "What is the difference between a stock and a bond?"
response, latency = client.generate_answer(query)

print(f"Response:\n{response}")
print(f"\nLatency: {latency:.4f} seconds")

Architecture & Components

1. QAService (qa_service.py)

Main entry point for handling chat completion requests.

• __init__(base_url: str): Sets up the underlying OpenAI client using the supplied endpoint URL.
• generate_answer(query: str, temperature: float = 0.1) -> tuple[str, float]: Formats system and user prompts, executes the request against the server, and returns a tuple containing:
  • response_content: The generated text response string or an error message.
  • latency: The execution duration in seconds.

2. PromptManager (prompt_manager.py)

Utility class responsible for loading and rendering prompt templates from client_prompts.yml.

• get_prompt(prompt_name: str) -> str: Fetches the raw template for a given key.
• format(prompt_name: str, **kwargs) -> str: Injects variable parameters (e.g., {input}) into template placeholders.

3. Prompt Configuration (client_prompts.yml)

Stores system instructions and user prompt templates:

• system_prompt: Sets the model role as a specialized financial advisor with strictly enforced boundaries.
• prompt_template: Wraps the user query with execution instructions.