🇮🇳 bharat-tiny-llm — Offline Hindi/Hinglish AI for Apple Silicon

bharat chat — an 880 MB Hindi/Hinglish assistant that runs fully offline. No API, no cloud, no internet.

Built by eulogik.

Install

pip install "bharat-tiny-llm[mlx]"   # Apple Silicon (recommended)
pip install "bharat-tiny-llm[torch]" # CPU/CUDA (loaders only)

Use

bharat chat                  # interactive REPL with history
bharat ask "नमस्ते!"         # one-shot question
bharat info                  # model + package info

First run downloads the 880 MB model from Hugging Face; after that everything is offline.

आप › Chai peete hain?
Chai peet hai, main ek assistant hoon.

Python API

from bharat_tiny_llm import chat

print(chat([{"role": "user", "content": "नमस्ते, आप कैसे हैं?"}], use_v2=True))

What's inside

Qwen2.5-1.5B + 300 injected Devanagari tokens ("Brahmi injection", −36.5% Hindi tokenization cost) + LoRA, quantized to 4-bit MLX. Measured benchmarks: bharat-tiny-llm benchmarks.

Notes & limitations

• Requires Apple Silicon for the CLI (mlx); Python chat() also works via the torch backend on other platforms.
• Don't pass system prompts — v2 was trained without system turns.
• Sampling defaults are canonical (temp 0.3, top-p 0.85, rep-penalty 1.25) — changing them can degrade output badly.
• Conversational quality is decent; factual/math reliability at 1.5B is limited.

Links

• 🤗 Model: https://huggingface.co/eulogik/Bharat-Tiny-LLM-v2-MLX
• 💻 Source: https://github.com/eulogik/Bharat-Tiny-LLM
• 📊 Benchmarks: https://github.com/eulogik/Bharat-Tiny-LLM/blob/main/docs/benchmarks_v2.md
• 🏢 Built by eulogik

License

Apache-2.0