kei-typing-assistant 0.1.0

AI-powered desktop typing assistant with real-time ghost-text completions, personalized via local embeddings and semantic example retrieval

Kei Typing Assistant

An AI-powered desktop typing assistant that provides real-time text completions as ghost text via a floating overlay window, accessible from any application. Learns your writing style over time for increasingly personalized suggestions.

Features

• Global hotkey opens a compact overlay window from any app; press it again to copy and close
• Streaming AI suggestions appear as faint ghost text while you type
• Accept a suggestion with Tab, dismiss with Esc
• Three AI providers: local LLM (llama.cpp, runs fully offline), MiniMax, and OpenRouter
• Hybrid mode: races the local model against a cloud provider and upgrades to the cloud result if it arrives first, within a configurable time window
• Model warm-up loads the local model in the background at startup so the first suggestion isn't slowed by a cold start
• Accepted suggestions are stored with semantic embeddings and retrieved as personalized few-shot context, with per-example search/shown counts tracked over time
• Manage stored examples (view, search, add, delete) from a dedicated GUI window
• Reusable, ordered context fragments — curated notes you can prepend to the AI system prompt — managed from their own GUI window
• Mid-word suppression skips AI requests while the cursor sits inside a word
• API keys stored securely in the system keychain
• SQLite storage with Alembic-managed schema migrations

Requirements

• Python 3.12+
• macOS 12+ (primary), Windows 10/11, Linux supported
• API key for MiniMax or OpenRouter (not required for the default local LLM provider)
• macOS: Accessibility permission required for global hotkey

Quick Start

1. Install

git clone https://github.com/l0kifs/kei-typing-assistant.git
cd kei-typing-assistant
uv sync

2. Configure (optional)

Create a .env file in the project root to override defaults:

KEI__AI_PROVIDER=local_llm      # local_llm | minimax | openrouter
KEI__HOTKEY=ctrl+alt+space      # global hotkey
KEI__FLAVOUR="Concise, formal"  # personalization hint for AI

All settings with defaults are documented in src/kei_typing_assistant/config/settings.py.

3. Add your API key (cloud providers only)

If using MiniMax or OpenRouter, open Settings from the tray icon and enter your API key. Keys are stored in the system keychain — never in plaintext. The default local LLM provider needs no API key; the model is downloaded from Hugging Face on first run.

4. Grant Accessibility permission (macOS)

System Settings → Privacy & Security → Accessibility — add the app or your terminal to allow global hotkey capture.

5. Run

uv run kei

The app runs in the system tray. Press the hotkey (Ctrl+Alt+Space by default) to open the overlay.

Usage

Action	Key
Open overlay	Ctrl+Alt+Space (configurable)
Accept suggestion	Tab
Copy text & close	Ctrl+Alt+Space again
Dismiss without copying	Esc

AI Providers

Provider	Setting value	Notes
Local LLM	local_llm (default)	Runs fully offline via llama.cpp; GGUF model auto-downloaded from Hugging Face; GPU-accelerated via Metal/CUDA where available
MiniMax	minimax	Cloud API, accessed via an Anthropic-compatible endpoint
OpenRouter	openrouter	Cloud API, accessed via an OpenAI-compatible endpoint; free-tier models available
Hybrid	enable hybrid_mode	Races the local model against a chosen cloud provider (MiniMax or OpenRouter) and swaps in the cloud result if it beats the local one within hybrid_upgrade_window_s

Settings Window

Tab	Contents
General	Writing-style ("flavour") hint injected into the AI system prompt
AI Model	Provider selection (Local LLM / MiniMax / OpenRouter / Hybrid) and per-provider configuration (model, API key, context window, GPU layers, sampling params)
Behaviour	Suggestion length, temperature, debounce delay, mid-word suppression, include-context-fragments toggle
Appearance	Theme, ghost-text opacity, font family/size, overlay position
Hotkeys	Global hotkey binding
Storage	Data directory location (models, database)

Two standalone windows, opened from the tray/settings menu, manage personalization data directly:

• Context Fragments — add, edit, reorder, search, and delete reusable prompt snippets prepended to the AI system prompt when enabled
• Suggestion Examples — browse, search, add, and delete the accepted (input → suggestion) pairs used for semantic few-shot retrieval, including how often each has been searched/shown

Configuration Reference

Setting	Default	Description
KEI__HOTKEY	ctrl+alt+space	Global trigger hotkey
KEI__AI_PROVIDER	local_llm	local_llm, minimax, or openrouter
KEI__AI_MAX_TOKENS	60	Max suggestion length
KEI__AI_TEMPERATURE	0.8	Sampling temperature
KEI__AI_DEBOUNCE_MS	300	Delay before AI request (ms)
KEI__MID_WORD_SUPPRESSION	false	Skip requests while the cursor is mid-word
KEI__INCLUDE_CONTEXT_FRAGMENTS	false	Prepend saved context fragments to the AI system prompt
KEI__FLAVOUR	``	Writing style hint for AI
KEI__THEME	system	system, dark, or light
KEI__HYBRID_MODE	false	Race local LLM against a cloud provider
KEI__HYBRID_UPGRADE_WINDOW_S	1.5	Seconds to wait for the cloud provider before keeping the local result
KEI__LOCAL_LLM_MODEL_ID	Qwen/Qwen2.5-3B-Instruct-GGUF	Hugging Face repo for the local model
KEI__LOCAL_LLM_N_GPU_LAYERS	-1	GPU layers to offload (-1 = all, 0 = CPU only)
KEI__SEMANTIC_EXAMPLES_TOP_K	5	Max similar accepted examples injected as few-shot context
KEI__DATA_DIR	~/.kei_typing_assistant	Base directory for models, database, and settings

Full list of settings, defaults, and validation ranges: src/kei_typing_assistant/config/settings.py.

Development

# Run tests
pytest

# Run tests with coverage
pytest --cov=src/kei_typing_assistant

# Lint & format
ruff check .
ruff format .

CLI Reference

kei-cli provides a command-line interface for testing suggestions, managing the semantic example store, local models, database migrations, and configuration.

uv run kei-cli [--verbose] COMMAND [ARGS]

Suggestion commands

Command	Description
suggest TEXT	Stream an AI suggestion for the given text
accept TEXT SUGGESTION	Save an accepted (input, suggestion) pair to the semantic example store
test [--text TEXT]	Full E2E flow: suggest → save accepted example

Typical workflow:

# 1. Get a suggestion
uv run kei-cli suggest "The weather today is"
#   → warm and sunny with clear skies

# 2. Accept it — embeds and saves for future retrieval
uv run kei-cli accept "The weather today is" "warm and sunny with clear skies"
#   ✓ Saved: 'The weather today is' → 'warm and sunny with clear skies'

# 3. Next suggest call retrieves similar stored examples as context
uv run kei-cli suggest "The weather this morning is"

suggest options: --flavour TEXT, --max-tokens INT test options: --text TEXT, --flavour TEXT, --max-tokens INT

Local model

uv run kei-cli model list             # List downloaded models
uv run kei-cli model status MODEL_ID  # Check if a model is downloaded
uv run kei-cli model delete MODEL_ID  # Delete a downloaded model from disk

Database

uv run kei-cli db upgrade   # Migrate the database to the latest schema revision
uv run kei-cli db current   # Show the currently applied schema revision

Configuration

uv run kei-cli config show      # Show current configuration

Architecture

The project follows Domain-Driven Design. See docs/ddd-architecture-rules.md for conventions and docs/BRD.md for the business requirements behind it.

src/kei_typing_assistant/
├── main.py          # Composition root: wiring, bootstrap, launch
├── config/          # Settings & logging
├── domains/         # Pure business logic (suggestion, context_fragments, local_model)
├── infrastructure/  # AI provider clients, SQLite database + Alembic migrations, keychain
└── entry_points/    # Qt GUI, hotkey listener, tray, CLI, background workers

Database schema changes are managed with Alembic; migrations run automatically at startup (kei-cli db upgrade for manual control).

Publishing

See docs/PUBLISHING.md. Releases are published to PyPI automatically via GitHub Actions on tag push. See CHANGELOG.md for release history.