vakforge

The decision layer for open voice AI.

vakforge helps you build a voice assistant from the data your company already has, and run it on your own servers. Before you spend anything on GPUs, it tells you what actually needs training. Usually that is less than you think.

• Bring any data: documents, FAQs, database tables, chat logs, CRM records, recorded calls.
• Open models only: nothing calls a hosted API. You own the result.
• Any language: through locale packs. Launching with English (US, UK, India) and Hinglish.

Status: pre-alpha. init, validate, inspect and the locale packs work today. See docs/ROADMAP.md.

Why vakforge exists

Open voice models are good enough today. Moshi, PersonaPlex, LFM2.5-Audio and Qwen-Omni can listen and talk, and strong open speech-to-text and text-to-speech models exist too. Yet most companies still pay per minute for a closed voice API.

That is not because the open models are worse. It is because nobody helps them answer four questions:

• Should we fine-tune, or is retrieval enough?
• Which model fits our language, our speed needs and our hardware?
• What data do we need, and are we allowed to train on our calls?
• How do we prove the result is better before a customer hears it?

vakforge answers them from your own data.

The hard part was never the models. It was the decision.

"Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty." Jensen Huang, NVIDIA, on X, 24 July 2026

vakforge is about that last word: owning your voice AI instead of renting it.

How it works

1. init creates a project folder for your language and market.
2. inspect reads your data folder and reports what is in it: languages, personal data, tables that could become tools, audio quality.
3. recommend decides what needs changing. Often the answer is retrieval and tools, with no training at all.
4. prepare, train, eval, serve are built for your project by a coding agent using the vakforge skill. It trains only what recommend asked for, and ships only if eval shows the result beats the base model.

Today the CLI covers init, validate, inspect and the locale packs. recommend is next.

How the decision is made

Every route ends the same way: test it against your own held-out data, and ship only if it beats the base model.

Each goal has its own fix, and most of them are not a full fine-tune. The detailed rules, with how much data each route needs and what hardware it takes, are in docs/DECISION_GUIDE.md.

The problem, for engineers

Open speech-to-speech models exist. Fine-tuning scripts exist for some of them. Evaluation tools and serving frameworks exist. What is missing is one path from "here is what my company knows" to "here is a voice assistant that handles my workflow, I can prove it is better than the base model, and my existing voice client can talk to it without a rewrite." Every team rebuilds that path, and most of them fine-tune when they should have used retrieval.

What's in the repo

1. A small library and command-line tool (pip install vakforge). No machine-learning dependencies, so it runs on any laptop. It holds the dataset format, the data checks, the data inspector, the decision rules and the locale packs.
2. An agent skill (skill/, in progress). Add it to Claude Code or another coding agent. The agent reads your data, follows the decision rules, and writes the training and serving code for your project. It checks every library it uses against the installed source first, so it does not guess at APIs.
3. Recipes (docs/RECIPES.md). Written-down, tested paths from an open base model to a running assistant. Only recipes someone has run end to end get listed.

Bring any data

Locale packs

The core works in any language. Everything that changes by language or country lives in a locale pack: how money, dates and phone numbers are written, which ID numbers count as personal data, the local privacy and call-recording rules, and which open models support the language. Adding a new market means adding a pack, not changing the core. See docs/LOCALE_PACKS.md, or run vakforge locales.

Recipes

"Duplex" means the assistant can listen while it talks, so callers can interrupt it naturally. Details in docs/RECIPES.md.

Connecting your app

Your assistant runs an open model on your own servers. Nothing calls OpenAI or any other hosted API.

To make switching easy, the server accepts the same WebSocket messages as OpenAI's Realtime API. If your app already talks to GPT Realtime, you change one URL. Tools like Pipecat, LiveKit and Twilio keep working as they are. More connection types can be added without touching the model:

Why English and Hinglish first

English has the strongest open voice models, so every recipe works for US, UK and Indian English. Hinglish is the hard test: people switch between Hindi and English mid-sentence, write Hindi in Roman or Devanagari script, and say amounts like "2 lakh rupees" on noisy phone lines. If vakforge handles that, adding another language is mostly filling in a new pack.

Quick start

pip install vakforge
vakforge init my-assistant --locale hi-Latn-IN
# put your documents, tables, chat exports and call recordings in my-assistant/data/raw/
vakforge inspect my-assistant/data/raw

inspect prints a summary and writes inspect.json. vakforge recommend, which reads it, is next.

Documentation

• docs/DECISION_GUIDE.md: what needs customizing, and when not to fine-tune
• docs/LOCALE_PACKS.md: what a locale pack contains, and how to add one
• docs/DATA_FORMAT.md: the dataset format
• docs/ARCHITECTURE.md: how the code fits together, with diagrams
• docs/RECIPES.md: per-model training recipes
• docs/EVALUATION.md: metrics, report format, per-locale benchmarks
• docs/DATA_ETHICS.md: consent, PII, licences, privacy law by region
• docs/ROADMAP.md: status
• CONTRIBUTING.md

Related projects (we build on these, not against them)

Unsloth, LLaMA-Factory, ms-swift, kyutai-labs/moshi-finetune, NVIDIA PersonaPlex, liquid-audio, Qwen-Omni, Pipecat, LiveKit Agents, vLLM-omni, UltraEval-Audio, AI4Bharat, Common Voice

Licence

Apache-2.0 for this code. Each open model keeps its own licence; see docs/RECIPES.md. Datasets you create with vakforge are yours. The consent records vakforge asks for are there to keep it that way.