LLMmodelHub 0.1.0

Turn any machine or Google Colab notebook into a one-line AI model server with an auto-generated public API.

LLMmodelHub

Turn any local machine or Google Colab notebook into a production-ready AI model server with a single line of code.

pip install LLMmodelHub

from LLMmodelHub import load_model

hub = load_model("TheBloke/Llama-2-7B-Chat-GGUF")

print(hub.local_url)    # http://0.0.0.0:5000
print(hub.public_url)   # https://xxxx.ngrok-free.app -- works out of the box

hub.chat()               # interactive terminal chat (or a widget UI in Colab)

Features

• One-line model loading — pulls models straight from Hugging Face, including quantized GGUF files for consumer hardware.
• LLM + VLM support — text models and vision-language models behind one API.
• Automatic REST API — an OpenAI-compatible /v1/chat/completions endpoint is spun up the moment the model loads.
• Public tunneling built in — pyngrok ships as a core dependency, so every load_model(...) call gets a public https://...ngrok-free.app URL automatically (falls back to a cloudflared quick tunnel if ngrok can't start). No separate ngrok install needed. For a stable, non-rate-limited URL, grab a free authtoken from ngrok.com and run from pyngrok import ngrok; ngrok.set_auth_token("...") once.
• Terminal chat UI — color-coded, with /image, /clear, /url commands.
• Colab widget UI — chat box, image upload, and URL display inside the notebook.
• Caching — downloaded models are cached locally and reused.
• Optional security — API key auth, CORS, and rate limiting.

Installation

pip install LLMmodelHub                 # core (API + downloader only)
pip install "LLMmodelHub[llm]"          # + transformers/torch for full HF models
pip install "LLMmodelHub[gguf]"         # + llama-cpp-python for GGUF models
pip install "LLMmodelHub[colab]"        # + ipywidgets for the notebook UI
pip install "LLMmodelHub[full]"         # everything

Usage

Load a full Hugging Face model

from LLMmodelHub import load_model

hub = load_model("meta-llama/Llama-2-7b-chat-hf", model_type="llm")
reply = hub.generate("Explain quantum computing in one sentence.")
print(reply)

Load a quantized GGUF model

hub = load_model("TheBloke/Llama-2-7B-Chat-GGUF", filename="llama-2-7b-chat.Q4_K_M.gguf")

Load a vision-language model

hub = load_model("Salesforce/blip2-opt-2.7b", model_type="vlm")
reply = hub.generate("What is in this image?", image_path="cat.jpg")

Call the REST API from anywhere

curl -X POST "$PUBLIC_URL/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello!"}]}'

Stop the server

hub.stop()

License

MIT