gracefox-llm-hub 0.1.4

统一的LLM调用库，支持多种LLM提供商（OpenAI、DeepSeek、通义千问、智谱GLM、百度文心、MiniCPM、Kimi、Gemini、Claude、Ollama、llama.cpp）

LLM Hub - Unified LLM Calling Library

A unified calling library supporting multiple LLM providers (International + Domestic Chinese AI), providing consistent API interfaces, command-line tools, and graphical interface.

✨ Features

• 🔌 Unified API - One interface for all LLM providers
• 🌍 Multi-Provider Support - OpenAI, DeepSeek, Kimi, Gemini, Claude, and more
• 🇨🇳 Domestic Chinese AI - Qwen (Tongyi Qianwen), GLM (Zhipu), ERNIE (Baidu), MiniCPM
• 💻 Local Deployment - Ollama, llama.cpp
• 🤔 Thinking Mode - Display reasoning process (DeepSeek support)
• 📊 Accurate Token Counting - Official tokenizers for each provider
• 🖥️ CLI & GUI - Command-line and graphical interfaces
• 📡 Streaming Output - Real-time response generation
• 💾 Config Management - JSON/YAML configs, profiles, environment variables

📋 Supported Providers

Provider	Type	Token Counter	Thinking Mode
OpenAI	Cloud	✅ tiktoken	❌
DeepSeek	Cloud	✅ deepseek-tokenizer	✅
Kimi (Moonshot)	Cloud	✅ API	❌
Google Gemini	Cloud	⚠️ Estimate	❌
Anthropic Claude	Cloud	⚠️ Estimate	❌
Qwen (Tongyi)	Cloud	✅ qwen-tokenizer	❌
GLM (Zhipu)	Cloud	✅ transformers/API	❌
ERNIE (Baidu)	Cloud	✅ transformers	❌
MiniCPM	Cloud/Local	✅ transformers	❌
Ollama	Local	✅ API tokenize	❌
llama.cpp	Local	✅ Subprocess	❌

🚀 Quick Start

Installation

# Basic installation
pip install gracefox-llm-hub

# Install with UI support
pip install gracefox-llm-hub[ui]

# Install with accurate token counting
pip install gracefox-llm-hub[tokenizers]

# Install all features
pip install gracefox-llm-hub[all]

Basic Usage

from llm_hub import LLMClient, LLMConfig, Provider

# Create client for DeepSeek
client = LLMClient(LLMConfig(
    provider=Provider.DEEPSEEK,
    model="deepseek-chat",
    api_key="your-api-key"
))

# Single turn chat
response = client.chat("What is artificial intelligence?")
print(response.content)

# Streaming output
for chunk in client.stream_chat("Tell me a story"):
    print(chunk, end="", flush=True)

Multi-turn Chat

from llm_hub import LLMClient, LLMConfig, Provider
from llm_hub.models.message import Conversation

client = LLMClient(LLMConfig(
    provider=Provider.OPENAI,
    model="gpt-3.5-turbo",
    api_key="your-api-key"
))

conv = Conversation()
conv.add_user("My name is Xiao Ming")
response = client.chat(conv)
conv.add_assistant(response.content)

conv.add_user("Do you remember my name?")
response = client.chat(conv)
print(response.content)  # Will remember "Xiao Ming"

Thinking Mode (DeepSeek)

from llm_hub import LLMClient, LLMConfig, Provider
from llm_hub.models.response import StreamEventType

config = LLMConfig(
    provider=Provider.DEEPSEEK,
    model="deepseek-chat",
    api_key="your-api-key",
    reasoning=True,      # Enable thinking mode
    show_thinking=True,  # Display thinking process
    stream=True
)

client = LLMClient(config)

for chunk in client.stream_chat("Solve 15 + 27 step by step"):
    if chunk.type == StreamEventType.THINKING:
        print(f"\n[Thinking] {chunk.delta}")
    elif chunk.type == StreamEventType.CONTENT:
        print(chunk.delta, end="", flush=True)

Token Counting

from llm_hub import LLMClient, LLMConfig, Provider
from llm_hub.utils.token_counter import get_token_counter

client = LLMClient(LLMConfig(
    provider=Provider.QWEN,
    model="qwen-turbo",
    api_key="your-api-key"
))

# Count tokens for text
tokens = client.count_tokens("Hello, 你好！")
print(f"Token count: {tokens}")

# Count conversation tokens
total = client.count_conversation_tokens(conversation)
print(f"Total tokens: {total}")

# Truncate by token limit
truncated = client.truncate_to_tokens(long_text, max_tokens=1000)

Domestic Chinese AI Examples

from llm_hub import LLMClient, LLMConfig, Provider

# Qwen (Tongyi Qianwen)
client = LLMClient(LLMConfig(
    provider=Provider.QWEN,
    model="qwen-turbo",
    api_key="your-dashscope-api-key"
))

# GLM (Zhipu)
client = LLMClient(LLMConfig(
    provider=Provider.GLM,
    model="glm-4-flash",
    api_key="your-zhipu-api-key"
))

# ERNIE (Baidu Wenxin)
client = LLMClient(LLMConfig(
    provider=Provider.ERNIE,
    model="ernie-3.5-8k",
    api_key="your-api-key",
    secret_key="your-secret-key"  # Required for Baidu
))

# MiniCPM (local via Ollama)
client = LLMClient(LLMConfig(
    provider=Provider.MINICPM,
    model="minicpm",
    ollama_url="http://localhost:11434"
))

🖥️ Command Line Interface

# Basic usage
llm-hub --provider deepseek --api-key YOUR_KEY "Hello"

# Use JSON config string
llm-hub --json '{"provider": "deepseek", "model": "deepseek-chat", "api_key": "xxx"}' "Hello"

# Use config file
llm-hub --config-file config.json "Hello"

# Use config profile
llm-hub --profile my-deepseek "Hello"

# Interactive mode
llm-hub --provider ollama --interactive

# Streaming output
llm-hub --provider openai --stream "Tell a story"

# List all config profiles
llm-hub --list-profiles

# Save current config as profile
llm-hub --provider deepseek --api-key xxx --save-profile my-ds

🖼️ Graphical User Interface

# Launch UI
llm-hub-ui

# Or
python run_ui.py

The UI supports:

• Provider switching (International + Domestic AI)
• Thinking mode display
• Real-time streaming output
• Token statistics
• Conversation export
• Config loading/saving

⚙️ Configuration

Configuration File (JSON)

{
    "provider": "deepseek",
    "model": "deepseek-chat",
    "api_key": "your-api-key",
    "temperature": 0.7,
    "max_tokens": 2000,
    "reasoning": true,
    "stream": true
}

Environment Variables

export LLM_PROVIDER=deepseek
export LLM_MODEL=deepseek-chat
export LLM_API_KEY=your-key
export LLM_TEMPERATURE=0.7

llm-hub "Hello"

Configuration Priority

Lowest to highest:

1. Default configuration
2. Environment variables
3. Config profile
4. Config file
5. JSON string
6. Command line arguments

📦 Project Structure

llm-hub/
├── llm_hub/
│   ├── core/           # Core functionality
│   ├── models/         # Data models
│   ├── providers/      # LLM provider implementations
│   ├── utils/          # Utilities (token counter, logger)
│   └── ui/             # GUI application
├── examples/           # Example code
├── tests/              # Unit tests
├── pyproject.toml      # Project configuration
└── README.md           # Documentation

🔧 Development

# Clone repository
git clone https://gitee.com/SteHub/llm-hub.git
cd llm-hub

# Install in development mode
pip install -e .[dev,all]

# Run tests
pytest

# Format code
black llm_hub/

# Run CLI
llm-hub --help

# Run UI
llm-hub-ui

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing)
5. Open a Pull Request

📄 License

GNU General Public License v3.0 - See LICENSE file for details

GPL-3.0 Core Requirements

• Freedom to Use - Anyone can freely use this software
• Source Code Availability - Source code must be provided when distributing
• Same License - Modified versions must be released under the same GPL-3.0 license
• Copyright Notice - Original copyright and license notices must be retained

🙏 Acknowledgements

• OpenAI - GPT models
• DeepSeek - DeepSeek chat models
• 阿里云 - Tongyi Qianwen (Qwen)
• 智谱AI - GLM models
• 百度 - ERNIE (Wenxin)
• Ollama - Local model runner
• llama.cpp - Local GGUF model runner

📧 Contact

• Author: GraceFox
• Email: [e-mail]
• Homepage: https://gitee.com/SteHub/llm-hub
• Issues: https://gitee.com/SteHub/llm-hub/issues

---

Quick Reference

Common Commands

# Install
pip install gracefox-llm-hub

# Run CLI
llm-hub --provider deepseek --api-key KEY "Hello"

# Run UI
llm-hub-ui

# Test token counting
python -c "from llm_hub.utils.token_counter import get_token_counter; c = get_token_counter('openai'); print(c.count('Hello'))"

Common Providers

Provider	Provider Value	Model Example
OpenAI	openai	gpt-3.5-turbo
DeepSeek	deepseek	deepseek-chat
Kimi	kimi	moonshot-v1-8k
Qwen	qwen	qwen-turbo
GLM	glm	glm-4-flash
ERNIE	ernie	ernie-3.5-8k
Ollama	ollama	llama2

---

⭐ Star this project on Gitee if you find it useful!