Yumi

One API to let AI call functions in any language, on any device.

Register a function. AI calls it. Python, Rust, Kotlin, Dart, C++, Swift, TypeScript, Go, Java, C# — same pattern everywhere.

Status: alpha. The core workflows are usable today, but APIs, generated templates, and UX may still change as the project stabilizes.

Quick Start

pip install yumi-agent       # or, from source: git clone … && pip install .
yumi --server                # first run walks you through provider/model setup
yumi --demo                  # in another terminal: launches Smart Home + Planner
yumi --chat                  # ask AI to control them in natural language

Connect your own app:

cd my_project
yumi --edge                  # interactive wizard: language, device name, Nexus or local/custom server
yumi --edge --lang python    # or jump straight to a language (also: typescript, swift, go, rust, kotlin, dart, java, csharp, cpp, ue5)

yumi --edge first lets you choose Yumi Nexus, Local / custom server, or Set up later. Local single-user servers need no Nexus login; Nexus supports browser sign-in or an account connection code. It scaffolds a yumi_tools/ directory. Edit the generated setup file, call its init function from your app entry point, and your functions appear as AI tools. Full walkthrough in Getting Started.

Demo

yumi --demo launches two independent Python GUIs at once:

• Smart Home — lights, TV, thermostat, coffee machine, locks (room cards + status)
• Planner — tkinter schedule app with a mini calendar and day timeline

Both windows are display-only. Open yumi --chat or yumi --ui and try:

Turn on the kitchen lights and add a "Cook dinner" event at 18:00 for 1 hour, category personal.

The demo requires a graphical desktop session. On Linux, install Tk first (sudo apt install python3-tk on Debian/Ubuntu). More example prompts are in Getting Started.

Same pattern, every language

The flow is the same in every language: run yumi --edge → implement tools in any module you like → import them in the generated setup file, register them, and define init_yumi / initYumi → call that init function once from your app's entry point. There is no required folder layout; only the imports in setup need to reach your functions.

Python

# my_app/tools.py
def analyze_data(path: str) -> str:
    """Load a CSV and return a short summary."""
    return "summary"

# yumi_tools/python/yumi_setup.py
from my_app.tools import analyze_data
from .yumi_sdk import YumiAgent

def init_yumi():
    agent = YumiAgent(edge_name="My Server")
    agent.register(analyze_data, "Analyze CSV at path and return a short summary")
    return agent

# Standalone script:  init_yumi().run()              (blocks until Ctrl+C)
# Embedded in an app: init_yumi().run_in_background() (returns immediately)

# your app entry point
from yumi_tools.python.yumi_setup import init_yumi
init_yumi()
# … rest of your program …

If you embed Yumi inside an already-installed Python package, from yumi.sdk import YumiAgent works directly without the yumi_tools/ tree.

TypeScript

// yumi_tools/typescript/yumiSetup.ts
import { YumiAgent } from "./yumi_sdk/src";
import { searchProducts } from "../src/catalog";

export function initYumi() {
  const agent = new YumiAgent({ edgeName: "My Web App" });
  agent.register({
    name: "searchProducts",
    description: "Search the product catalog",
    handler: async (args) => searchProducts(args.string("query") ?? ""),
  });
  agent.runInBackground();
  return agent;
}

Other languages

C++, Swift, Go, Java, C#, Rust, Kotlin, Dart, and UE5 follow the same pattern with idiomatic syntax. See the Edge Tools Guide for full code samples in each language.

How It Works

flowchart TB
  subgraph ai [AI Brain]
    LLM[LLM Provider]
    Server[Yumi Server]
  end
  subgraph devices [Your Devices — Any Language]
    RPi["Raspberry Pi · C++"]
    Phone["iPhone · Swift"]
    Desktop["Desktop App · Python"]
    Web["Web App · TypeScript"]
    IoT["IoT Gateway · Go"]
  end
  LLM <--> Server
  Server <-->|WebSocket| RPi
  Server <-->|WebSocket| Phone
  Server <-->|WebSocket| Desktop
  Server <-->|WebSocket| Web
  Server <-->|WebSocket| IoT

Your app connects to the Yumi server over WebSocket and registers functions as tools. The LLM sees them alongside server-side tools and calls whichever it needs. Results flow back through the same connection.

Main Commands

Optional Integrations

• Telegram — chat with Yumi from a Telegram bot. Get a token from @BotFather, then run yumi --server --telegram (single machine) or yumi --telegram (bot only). Token, allowlist, and timer-push details: Configuration → Telegram.
• Discord — chat with Yumi from a Discord bot. Create an application + bot in the Discord Developer Portal, enable the Message Content intent, then run yumi --server --discord (single machine) or yumi --discord (bot only). Token, allowlist, and timer-push details: Configuration → Discord.
• LINE — chat from LINE via the Messaging API webhook. Run yumi --server --line (single machine, default port 8788) or yumi --line (webhook sidecar only). Credentials and webhook setup: Configuration → LINE.
• Voice — talk to Yumi through your microphone. Say the wake word ("hi yumi") and Yumi transcribes the rest of your sentence and runs it as a chat turn. Transcription runs locally with Whisper, or with no model download through a cloud provider that reuses your existing key — openai, gemini, grok, or dashscope (Qwen3-ASR). Coexists with Telegram / --chat / --ui so the same Yumi instance can listen and type at once, and recent voice/Telegram/CLI turns are merged into each prompt. Mic capture needs a Picovoice access key. Setup: Configuration → Voice.
• Spoken replies (TTS) — Yumi can talk back. Pick a backend in yumi --setup: system (Windows SAPI, macOS say, or Linux espeak), openai (OpenAI TTS, reuses your openai_api_key), gemini (Gemini TTS, reuses your gemini_api_key), grok (xAI Grok TTS, reuses your grok_api_key), dashscope (Qwen3-TTS via the DashScope API), or qwen (Qwen3-TTS run locally on a GPU via the optional [tts-local] extra). In voice mode replies are spoken automatically; on Telegram / Discord, /voice on (!voice on) switches a chat to audio replies. Test any time with yumi --speak "hello".

Supported Providers

You can mix providers — for example OpenAI for chat and Ollama for embeddings.

Edge SDKs

Documentation

How Yumi Differs

Yumi is not another Python-only LLM chaining library. It ships a runnable server, terminal UI, and web UI, plus first-class edge tool hosts across eleven languages. The focus is on device-side tool execution: your game, phone app, IoT sensor, or desktop program exposes functions, and the AI calls them directly in your process.

Core Scope

This package (yumi-agent) is the open-source, self-hosted core — run your own server at home. You chat with it remotely through Telegram, LINE, or Discord bridges, while edge devices register tools on the same machine or your LAN. No account scoping, no quotas — a complete single-user agent on your own hardware.

Use it locally through the terminal chat or web UI, connect messaging bridges when you want remote chat, and add your own apps/devices through Edge SDKs. The API is local-first and single-user by default, so keep it on localhost or a trusted network unless you put your own access controls in front.

License

Apache License 2.0 — see LICENSE and NOTICE.

Contributing · Security · Changelog · Code of Conduct