locallab 0.4.48

LocalLab: Run language models locally or in Google Collab with a friendly API

🚀 LocalLab: Your Personal AI Lab

LocalLab lets you run AI language models on your computer or Google Colab - no cloud services needed! Think of it as having ChatGPT-like capabilities right on your machine.

🤔 What is LocalLab?

LocalLab consists of two parts working together:

graph LR
    A[LocalLab Server] -->|Runs| B[AI Models]
    C[Your Code] -->|Uses| D[LocalLab Client] -->|Talks to| A

The Server (Your AI Engine)

Think of the server as your personal AI engine. It:

• Downloads and runs AI models on your computer
• Manages memory and resources automatically
• Optimizes performance based on your hardware
• Provides a simple API for accessing models

You can run it:

• On your computer (local mode)
• On Google Colab (free GPU mode)

The Client (Your AI Controller)

The client is how your code talks to the AI. It:

• Connects to your LocalLab server
• Sends requests for text generation
• Handles chat conversations
• Processes multiple requests at once
• Streams responses in real-time

✨ How It Works Together

When you use LocalLab:

1. Server Setup

   from locallab import start_server
   start_server()  # Server starts and loads AI model
   

2. Client Connection

   from locallab.client import LocalLabClient
   server_url = "http://localhost:8000" # or "https://your-ngrok-url.ngrok.app"
   client = LocalLabClient(server_url)
   

3. AI Interaction

   # Your code sends requests through the client
   # Async usage
   response = await client.generate("Write a story")
   print(response)  # Server processes and returns AI response
   
   # Or sync usage (New!)
   response = client.generate("Write a story")
   print(response)  # Same result, no async/await needed!
   

💡 Quick Examples

# Generate text (async or sync)
response = await client.generate("Hello!")  # Async
response = client.generate("Hello!")        # Sync (New!)

# Chat with AI (async or sync)
response = await client.chat([              # Async
    {"role": "user", "content": "Hi!"}
])
response = client.chat([                    # Sync (New!)
    {"role": "user", "content": "Hi!"}
])

# Process multiple prompts (async or sync)
responses = await client.batch_generate([   # Async
    "Write a joke",
    "Tell a story"
])
responses = client.batch_generate([         # Sync (New!)
    "Write a joke",
    "Tell a story"
])

# Context manager support (New!)
with LocalLabClient("http://localhost:8000") as client:
    response = client.generate("Hello!")    # Auto-closes when done

➡️ See More Examples

💻 Requirements

Local Computer:

• Python 3.8+
• 4GB RAM minimum
• GPU optional (but recommended)

Google Colab:

• Just a Google account!
• Free tier works fine

📚 Getting Started

1. Choose Your Path

New to AI/Programming?

1. Start with our Getting Started Guide
2. Try the Basic Examples
3. Join our Community

Developer?

1. Check API Reference
2. See Client Libraries
3. Read Advanced Features

2. Read the Docs

Our Documentation Guide will help you:

• Understand LocalLab's features
• Learn best practices
• Find solutions to common issues
• Master advanced features

🌟 Features

• Easy Setup: Just pip install and run
• Multiple Models: Use any Hugging Face model
• Resource Efficient: Automatic optimization
• Privacy First: All local, no data sent to cloud
• Free GPU: Google Colab integration
• Unified Client API: Works with or without async/await (New!)
• Automatic Resource Management: Sessions close automatically (New!)

➡️ See All Features

🔍 Need Help?

• Check FAQ
• Visit Troubleshooting
• Ask in Discussions

📖 Additional Resources

• Contributing Guide
• Changelog
• License

---

Made with ❤️ by Utkarsh Tiwari GitHub • Twitter • LinkedIn