lite-llm-client 0.1.0

A small example package

Lite LLM Client

This project made for very light llm client. the main idea is do not use any llm client library.

setup

How to pass API_KEY

1. use parameter of LLMConfig

LiteLLMClient(OpenAIConfig(api_key="YOUR API KEY"))

2. use .env
   • rename .env_example to .env
   • replace YOUR KEY to real api_key

OPENAI_API_KEY=YOUR KEY
ANTHROPIC_API_KEY=YOUR KEY
GEMINI_API_KEY=YOUR KEY

Known issue

• gemini path may not stable. guide code has /v1beta/.... sometimes gemini returns http 500 error

Roadmap

• 2024-07-21 support OpenAI
• 2024-07-25 support Anthropic
• 2024-07-27 add options for inference
• 2024-07-28 support Gemini
• 2024-07-30 support streaming (OpenAI). simple SSE implement.
• 2024-07-31 support streaming (Anthropic).
• 2024-08-01 support streaming (Gemini). unstable google gemini.
• support multimodal (image and text)

Reference

• OpenAI REST API
• Gemini REST API