Termux-BitNet

Production 1.58-bit (i2_s) BitNet On-Device Inference SDK with Native Vulkan Compute GPU Acceleration for Android Termux & ARM64.

Empirical Real-Device Benchmarks (Microsoft BitNet b1.58 2B-4T)

1. Overview & Architecture

termux-bitnet is an optimized on-device inference engine and dual SDK (Python & Node.js) engineered for running 1.58-bit quantized Large Language Models (BitNet b1.58) natively on Android Termux, ARM64 mobile processors, and edge devices.

[Python Application / CLI]        [Node.js / TypeScript App]
       │                                     │
       ▼ (BitNetEngine / ctypes)             ▼ (BitNetEngine / FFI)
[termux-bitnet Python SDK]        [termux-bitnet npm Thin Gateway]
       │                                     │
       └──────────────────┬──────────────────┘
                          │
             ┌────────────┴────────────┐
             ▼                         ▼
[Native C++17 BitNet Core]    [AMEVA-Runtime Vulkan Engine]
       │ (ARM NEON DotProd)              │ (SPIR-V Compute Shaders)
       ▼                                 ▼
[ARM64 CPU: 4 Big Cores]      [Mobile GPU: Adreno 830 / Mali-G68]

2. Installation & Verification

2.1 Python Package with Vulkan GPU Acceleration (PyPI)

# Inside Android Termux (prerequisites: clang cmake python openblas)
pkg update && pkg install -y clang cmake python openblas

# Install Termux-BitNet and AMEVA-Runtime for Vulkan GPU acceleration
pip install termux-bitnet ameva-runtime

2.2 Node.js / TypeScript Thin Gateway (npm)

# Global CLI installation (Provides 'termux-bitnet-js' command)
npm install -g termux-bitnet @ameva/runtime

2.3 Automated Installer (Precompiled Binary or Fast Native Build)

curl -sL https://raw.githubusercontent.com/uno-km/termux-bitnet/main/install.sh | bash

3. CLI Usage

3.1 Python CLI (termux-bitnet)

# 1. Hardware Diagnostic (ARM NEON, DotProd SIMD & Vulkan GPU Verification)
termux-bitnet info

# 2. Download Model with HTTP Range Resume Support
termux-bitnet download bitnet-2b

# 3. Run On-Device Inference with Native Vulkan GPU Acceleration
termux-bitnet run -m ~/.cache/termux-bitnet/models/bitnet-2b-ggml-model-i2_s.gguf \
  -p "Explain quantum computing in one sentence." \
  --device gpu -t 4 -c 2048 -n 64 --temp 0.7 --top-p 0.95

3.2 Node.js CLI (termux-bitnet-js)

# 1. Hardware Diagnostic
termux-bitnet-js info

# 2. Run Inference via Node.js Gateway with GPU backend
termux-bitnet-js run -m ~/.cache/termux-bitnet/models/bitnet-2b-ggml-model-i2_s.gguf \
  -p "Explain quantum computing in one sentence." -d gpu -t 4 -n 64

4. Programmatic API

4.1 Python SDK (with Vulkan GPU Mode)

from termux_bitnet import BitNetEngine, BitNetConfig

# 1. Configure Engine Parameters with Vulkan GPU Acceleration
config = BitNetConfig(
    model_path="~/.cache/termux-bitnet/models/bitnet-2b-ggml-model-i2_s.gguf",
    device="gpu",          # "gpu" (Vulkan via AMEVA-Runtime), "cpu", or "auto"
    n_threads=4,
    temperature=0.7,
    top_p=0.95,
    top_k=40,
    min_p=0.05,
    repeat_penalty=1.15,
)

# 2. Stream Generation with Dynamic Parameters
with BitNetEngine(config) as engine:
    print("[Prompt]: Write a Python palindrome check function")
    print("[Response]: ", end="", flush=True)
    for token in engine.generate_stream("Write a Python palindrome check function:"):
        print(token, end="", flush=True)
    print()
    metrics = engine.get_last_metrics()
    print(f"Speed: {metrics.tokens_per_second:.2f} tok/s, Prompt tokens: {metrics.prompt_tokens}")

4.2 Node.js & TypeScript SDK

const { createEngine } = require('termux-bitnet');

async function main() {
  const engine = createEngine({
    modelPath: '~/.cache/termux-bitnet/models/bitnet-2b-ggml-model-i2_s.gguf',
    device: 'gpu',         // 'gpu' (Vulkan), 'cpu', or 'auto'
    threads: 4,
    temperature: 0.7,
    topP: 0.95,
  });

  console.log('[Prompt]: Explain quantum computing in one sentence');
  console.log('[Response]: ');

  await engine.generateStream(
    'Explain quantum computing in one sentence',
    64,
    (token) => {
      process.stdout.write(token);
    }
  );
  console.log('\n');
}

main();

4.3 Direct AMEVA-Runtime Adapter API

from ameva_runtime.adapters.bitnet import BitnetAdapter

adapter = BitnetAdapter()
adapter.load_model("~/.cache/termux-bitnet/models/bitnet-2b-ggml-model-i2_s.gguf")

for token in adapter.generate("Explain quantum computing in one sentence:"):
    print(token, end="", flush=True)
print()

5. Configuration Parameter Matrix (BitNetConfig)

6. Official Documentation & Specifications

• Official Documentation Site: https://uno-km.vercel.app/lib/bitnet/
• AI Agent Context Feed: llms.txt
• Full Technical Specification: llms-full.txt

7. License & Foundation

Released under the Apache License 2.0.
Engineered under the AMEVA Open-Source Foundation (AOSF) & uno-km ecosystem.