nexgen-sdk 0.1.2

A custom Python SDK for interacting with AI models, compatible with OpenAI's API format

AI SDK Python

A comprehensive Python SDK for interacting with AI models, compatible with OpenAI's API format and supporting multiple providers.

Features

• 🤖 Multiple Provider Support: OpenAI, LiteLLM, and VLLM providers
• 🔄 Streaming Support: Real-time streaming responses with event handling
• 🛡️ Error Handling: Comprehensive error handling with custom exceptions
• 🔧 Flexible Configuration: Environment variables, config files, and programmatic setup
• 📦 Easy Integration: Simple, intuitive API design
• 🎯 Type Safety: Full type hints support

Installation

pip install nexgen-sdk

For development:

pip install nexgen-sdk[dev]

Quick Start

Basic Usage

from ai_sdk import AISDKClient

# Initialize client
client = AISDKClient(
    api_key="your-api-key",
    provider="openai"  # or "vllm", "litellm"
)

# Create chat completion
response = client.chat().create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Hello, world!"}
    ]
)

print(response['choices'][0]['message']['content'])

Streaming

# Stream responses in real-time
response = client.chat().create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)

for chunk in response:
    if chunk['choices'][0]['delta'].get('content'):
        print(chunk['choices'][0]['delta']['content'], end='', flush=True)

Provider-Specific Examples

VLLM Provider (requires base_url)

# Local vLLM deployment
client = AISDKClient(
    provider="vllm",
    base_url="http://localhost:8000",  # Required for vllm
    api_key="your-api-key"
)

# Custom vLLM endpoint
client = AISDKClient(
    provider="vllm",
    base_url="https://your-vllm-endpoint.com",  # Required
    api_key="your-api-key"
)

# New format supports complex user content objects
response = client.chat().create(
    system_prompt="You are a helpful AI assistant",
    messages=[
        {
            "role": "user",
            "content": {
                "query": "What is Python?",
                "detail_level": "intermediate",
                "include_examples": True
            }
        }
    ],
    model="your-model-name"
)

Configuration

Environment Variables

export AI_SDK_API_KEY="your-api-key"
export AI_SDK_BASE_URL="https://api.openai.com/v1"
export AI_SDK_TIMEOUT="30.0"

Builder Pattern

client = (AISDKClient.builder()
           .with_provider("vllm")
           .with_api_key("your-key")
           .with_base_url("https://your-vllm-endpoint")
           .with_timeout(60.0)
           .build())

Providers

OpenAI Provider

• Base URL: https://api.openai.com/v1
• Models: GPT-3.5, GPT-4, and all OpenAI models
• Features: Chat, completions, embeddings

VLLM Provider

• Required Configuration: base_url parameter is mandatory
• Custom Endpoints: Supports any VLLM deployment (local, cloud, custom)
• New Format: Enhanced message format with complex user content
• Streaming: Optimized real-time streaming

Note: When using provider="vllm", you must specify base_url pointing to your vLLM deployment.

LiteLLM Provider

• Unified Interface: Access 100+ models through LiteLLM
• Model Support: OpenAI, Anthropic, Cohere, and more
• Fallback: Automatic fallback between providers

API Reference

Client

• AISDKClient() - Main client class
• chat() - Access chat completions
• completions() - Access text completions
• embeddings() - Access embeddings

Chat Completions

• create() - Create chat completion or stream
• Parameters: messages, model, temperature, stream, etc.

Error Handling

from ai_sdk.exceptions import (
    AISDKException,
    APIException,
    AuthenticationException,
    RateLimitException
)

try:
    response = client.chat().create(...)
except AuthenticationException:
    print("Invalid API key")
except RateLimitException:
    print("Rate limit exceeded")
except AISDKException as e:
    print(f"SDK Error: {e}")

Development

Setup

git clone https://github.com/Dhruv1969Karnwal/ai-sdk-python
cd ai-sdk-python
pip install -e ".[dev]"

Running Tests

pytest

Code Formatting

black ai_sdk
flake8 ai_sdk
mypy ai_sdk

Examples

See the examples/ directory for comprehensive usage examples:

• vllm_examples.py - Complete VLLM provider examples with different deployments
• new_format_example.py - VLLM new format examples
• debug_streaming.py - Streaming debug examples
• test_clean_streaming.py - Clean streaming tests

VLLM Quick Examples

# Local vLLM deployment
client = AISDKClient(
    provider="vllm",
    base_url="http://localhost:8000",
    api_key="your-key"
)

# Custom endpoint
client = AISDKClient(
    provider="vllm",
    base_url="https://your-vllm-endpoint.com",
    api_key="your-key"
)

# Missing base_url will show helpful error message
# client = AISDKClient(provider="vllm", api_key="your-key")  # Error!

## Requirements

- Python 3.7+
- httpx>=0.23.0
- typing_extensions>=3.7.4 (Python < 3.8)

### Optional Dependencies
- litellm>=1.0.0 (for LiteLLM provider)

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Support

- 📖 [Documentation](https://ai-sdk.readthedocs.io)
- 🐛 [Issues](https://github.com/Dhruv1969Karnwal/ai-sdk-python/issues)
- 💬 [Discussions](https://github.com/Dhruv1969Karnwal/ai-sdk-python/discussions)

## Contributing

Contributions are welcome! Please see our [Contributing Guide](CONTRIBUTING.md) for details.

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for version history.