locallab 0.5.1

LocalLab: Run language models locally or in Google Collab with a friendly API

🚀 LocalLab: Your Personal AI Lab

Run powerful AI language models on your own computer or Google Colab - no cloud services needed! Think of it as having ChatGPT-like capabilities right on your machine.

🤔 What is LocalLab?

LocalLab brings AI to your fingertips with two key components:

graph TD
    A[Your Code] -->|Uses| B[LocalLab Client]
    B -->|Talks to| C[LocalLab Server]
    C -->|Runs| D[AI Models]
    C -->|Manages| E[Memory & Resources]
    C -->|Optimizes| F[Performance]

🎯 Key Features

📦 Easy Setup         🔒 Privacy First       🎮 Free GPU Access
🤖 Multiple Models    💾 Memory Efficient    🔄 Auto-Optimization
🌐 Local or Colab    ⚡ Fast Response       🔧 Simple API

🌟 Two Ways to Run

1. On Your Computer (Local Mode)

   💻 Your Computer
   └── 🚀 LocalLab Server
       └── 🤖 AI Model
           └── 🔧 Auto-optimization
   

2. On Google Colab (Free GPU Mode)

   ☁️ Google Colab
   └── 🎮 Free GPU
       └── 🚀 LocalLab Server
           └── 🤖 AI Model
               └── ⚡ GPU Acceleration
   

📦 Installation & Setup

1. Install Required Packages

# Install both server and client packages
pip install locallab locallab-client

2. Configure the Server (Recommended)

# Run interactive configuration
locallab config

# This will help you set up:
# - Model selection
# - Memory optimizations
# - GPU settings
# - System resources

3. Start the Server

# Start with saved configuration
locallab start

# Or start with specific options
locallab start --model microsoft/phi-2 --quantize --quantize-type int8

💡 Basic Usage

Synchronous Usage (Easier for Beginners)

from locallab_client import SyncLocalLabClient

# Connect to server
client = SyncLocalLabClient("http://localhost:8000")

try:
    print("Generating text...")
    # Generate text
    response = client.generate("Write a story")
    print(response)

    print("Streaming responses...")
    # Stream responses
    for token in client.stream_generate("Tell me a story"):
       print(token, end="", flush=True)

    print("Chat responses...")
    # Chat with AI
    response = client.chat([
        {"role": "system", "content": "You are helpful."},
        {"role": "user", "content": "Hello!"}
    ])
    print(response.choices[0]["message"]["content"])

finally:
    # Always close the client
    client.close()

Asynchronous Usage (For Advanced Users)

import asyncio
from locallab_client import LocalLabClient

async def main():
    # Connect to server
    client = LocalLabClient("http://localhost:8000")
    
    try:
        print("Generating text...")
        # Generate text
        response = await client.generate("Write a story")
        print(response)

        print("Streaming responses...")
        # Stream responses
        async for token in client.stream_generate("Tell me a story"):
            print(token, end="", flush=True)

        print("\nChatting with AI...")
        # Chat with AI
        response = await client.chat([
            {"role": "system", "content": "You are helpful."},
            {"role": "user", "content": "Hello!"}
        ])
        # Extracting Content
        content = response['choices'][0]['message']['content']
        print(content)
    finally:
        # Always close the client
        await client.close()

# Run the async function
asyncio.run(main())

🌐 Google Colab Usage

Run LocalLab on Google's free GPUs:

# 1. Install packages
!pip install locallab locallab-client

# 2. Configure with CLI (notice the ! prefix)
!locallab config

# 3. Start server with CLI
!locallab start --use-ngrok

# 4. Connect client (Locally)
from locallab_client import LocalLabClient
client = LocalLabClient("https://your-server-ngrok-url.app")
response = await client.generate("Hello!")

💻 Requirements

Local Computer

• Python 3.8+
• 4GB RAM minimum (8GB+ recommended)
• GPU optional but recommended
• Internet connection for downloading models

Google Colab

• Just a Google account!
• Free tier works fine

🌟 Features

• Easy Setup: Just pip install and run
• Multiple Models: Use any Hugging Face model
• Resource Efficient: Automatic optimization
• Privacy First: All local, no data sent to cloud
• Free GPU: Google Colab integration
• Flexible Client API: Both async and sync clients available
• Automatic Resource Management: Sessions close automatically

➡️ See All Features

📚 Documentation

Getting Started

1. Installation Guide
2. Basic Examples
3. CLI Usage

Advanced Topics

1. API Reference
2. Client Libraries
3. Advanced Features
4. Performance Guide

Deployment

1. Local Setup
2. Google Colab Guide

🔍 Need Help?

• Check FAQ
• Visit Troubleshooting
• Ask in Discussions

📖 Additional Resources

• Contributing Guide
• Changelog
• License

🌟 Star Us!

If you find LocalLab helpful, please star our repository! It helps others discover the project.

---

Made with ❤️ by Utkarsh Tiwari GitHub • Twitter • LinkedIn