lmLite

Package version: 0.9.0

A Python bridge to Transformers.js for JupyterLite — run language models and embedding workflows entirely in your browser.

Quick Start

Try online (no installation required):

• Launch with xeus-python (recommended)
• Launch with Pyodide

Basic usage:

from lmlite import LLM

# Instruction-tuned chat with SmolLM2
llm = await LLM.create(
    generator_model="HuggingFaceTB/SmolLM2-360M-Instruct",
    backend="wasm",
    dtype="q4",
)
response = await llm.chat([
    {"role": "user", "content": "Explain embeddings in simple terms."}
])
print(response)

# Text completion with GPT-2
llm = await LLM.create(generator_model="gpt2")
text = await llm.generate("Python is a great language because")
print(text)

# Embeddings and similarity search
llm = await LLM.create(embedding_model="all-MiniLM-L6-v2")
embedding = await llm.embed("Hello world")

docs = ["JupyterLite runs in the browser.", "Python is widely used for ML."]
results = await llm.similarity_search("Where does JupyterLite run?", docs)
print(results)

Features

• Browser-native: Run LLMs entirely in the browser, no server required
• Instruction-tuned models: Chat and instruction-following with browser-compatible models such as SmolLM2-360M-Instruct.
• Composable workflows: Build simple RAG pipelines and conversational interfaces with memory using Python.
• Privacy-first: All computation happens locally in your browser
• Zero setup: No installation or API keys for hosted demos
• Multiple backends: WebAssembly (WASM) or WebGPU
• Multiple models: GPT-2, DistilGPT-2, TinyLlama, and more via Transformers.js
• Embeddings & classification: Sentence vectors and text classification
• Task pipelines: Zero-shot classification, speech recognition and translation via LLM.pipeline(task, model=...)
• Model caching: Browser-cached downloads, portable ZIP export/import, and cache inspection before loading (LLM.model_info(), cached_models(), per-model clear_cache()) with Python progress callbacks
• Pythonic API: Clean, async Python interface

What is lmLite?

lmLite is a lightweight Python-to-JavaScript bridge for local LLM inference in the browser, designed for JupyterLite notebooks. It provides:

• Async Python API for text generation, embeddings, classification, similarity search, and additional Transformers.js task pipelines
• Model loading, caching, and portable ZIP workflows
• Independent backend/dtype controls for embeddings and generation

Inference is powered by Transformers.js and ONNX Runtime. lmLite is not a new model engine or native Python inference backend. Its similarity search is an in-memory cosine ranking helper, not a vector database.

Best suited for: Teaching, interactive experiments, and browser-local prototypes where avoiding remote inference services matters.

Note: Installing the Python package on desktop does not enable desktop inference. LLM.create() requires a supported browser-based Python environment (xeus-python or the JupyterLite Pyodide kernel) with JavaScript interoperability. See docs/RUNTIME.md for details.

Feedback and Contributions

lmLite is an experimental project exploring Python access to browser-native machine learning through Transformers.js.

Feedback is particularly welcome on:

• The usefulness and design of the asynchronous Python API.
• Additional Transformers.js tasks and capabilities to expose.
• Model compatibility, performance, and browser runtime limitations.
• Potential applications beyond JupyterLite.

Please share suggestions, bug reports, and use cases through GitHub Issues.

AI-Assisted Development

Parts of this project were developed with the assistance of generative AI tools. The maintainer is responsible for the architecture, review, testing, and published code.

AI-assisted contributions are welcome, provided contributors understand, review, and test their changes and take responsibility for their submissions.

Documentation

• Getting Started - Installation, development setup, and testing
• Usage Guide - Detailed API examples and configuration
• Model Guide - Recommended models and dtype selection
• Runtime & Integration - Platform support and integration boundaries
• Glossary - Key concepts and terminology

Interactive Notebooks

Try lmLite directly in your browser, without installing anything.

• Basic demo — Text generation, embeddings, classification, and similarity search.
• Task pipelines — Zero-shot classification, translation and speech recognition with LLM.pipeline().
• Model info and cache manager — Download size and cache status before loading, loading progress from Python, and a widget to list, load and delete cached models.
• LLM orchestration — A simple retrieval-augmented generation (RAG) workflow and interactive multi-turn chat with conversation memory and streamed, stoppable answers, using SmolLM2-360M-Instruct (WASM/q4) and MiniLM embeddings (WASM/q8).
• Streaming chat — Print a chat answer as it is generated, stop early with break, and cancel a running generation with cancel(), using SmolLM2-135M-Instruct (WASM).
• Token probabilities — Stream each generated token with its probability, plus the runner-up token at every step, using SmolLM2-135M-Instruct (WASM).
• Tokenizers — Load only the tokenizers of GPT-2, multilingual BERT and POTION (no weights), compare how they split text, and count tokens synchronously with LLM.tokenizer().

These examples run locally in the browser using Transformers.js.

How It Works

┌─────────────────────────────────────┐
│   Python (JupyterLite)              │
│   ├─ xeus-python (recommended)      │
│   └─ pyodide (alternative)          │
│                                     │
│   from lmlite import LLM            │
│   llm = await LLM.create()          │
│   text = await llm.generate(...)    │
└─────────────┬───────────────────────┘
              │ Bridge (xeus-python: pjs / Pyodide: pyodide.ffi)
┌─────────────▼───────────────────────┐
│   JavaScript (Browser)              │
│   Transformers.js + ONNX Runtime    │
│   ├─ WebAssembly (WASM)             │
│   └─ WebGPU (optional)              │
└─────────────────────────────────────┘

License

Apache License 2.0. See LICENSE for details.

Dependencies: Transformers.js v4 (Apache 2.0)

Note: Individual model licenses may vary. Check model cards on HuggingFace before use.

Citation

@software{lmlite2026,
  author = {Decitre, Emmanuel},
  title = {lmLite: Browser-native LLM orchestration in JupyterLite},
  year = {2026},
  url = {https://github.com/decitre/lmlite}
}

Acknowledgements

lmLite builds on several open-source projects:

• Transformers.js — browser-native model inference.
• JupyterLite — the browser-based Jupyter environment.
• xeus-python and Pyodide — Python execution in the browser.

Thanks to the developers and maintainers of these projects and the broader Jupyter and Hugging Face communities.