vakforge

The decision layer for open voice AI.

vakforge helps you build a voice assistant from the data your company already has, and run it on your own servers. Before you spend anything on GPUs, it tells you what actually needs training. Usually that is less than you think.

• Bring the data you have: documents, FAQs, database tables, chat logs, CRM exports, recorded calls. PDF, Word and Excel files are listed as skipped until text extraction lands; export them to text or CSV for now.
• Open models only: the assistant itself runs on your servers and never calls a hosted API. (Generating synthetic training dialogues may use a provider you choose, only after you approve it.)
• Any language: through locale packs. Launching with English (US, UK, India) and Hinglish.

Status: pre-alpha. init, inspect, recommend, report, validate and the locale packs work today. The agent skill that writes the rest of the path is written but has not yet been run end to end on a real project. See docs/ROADMAP.md.

Try it

pip install vakforge
vakforge init my-assistant --locale en-US       # or en-GB, en-IN, hi-Latn-IN
cd my-assistant
# copy your files into data/raw/, then:
vakforge inspect data/raw -o runs/inspect.json
vakforge recommend runs/inspect.json -o runs/recommend.json
vakforge report runs/inspect.json runs/recommend.json -o runs/report.html   # to share or save as PDF

inspect reports what is in the folder: languages, personal data (in table columns too), tables that could become tools, and every file it skipped and why. recommend says what each source needs, and whether anything is worth fine-tuning yet. Nothing leaves your machine. The reports go in runs/, which the project's .gitignore keeps out of git: inspect.json names your files and columns, and can name the people in a chat export.

No data to hand? examples/hinglish-shop is a small synthetic project with both reports committed next to it.

Why vakforge exists

Open voice models are good enough today. Moshi, PersonaPlex, LFM2.5-Audio and Qwen-Omni can listen and talk, and strong open speech-to-text and text-to-speech models exist too. Yet most companies still pay per minute for a closed voice API.

That is not because the open models are worse. It is because nobody helps them answer four questions:

• Should we fine-tune, or is retrieval enough?
• Which model fits our language, our speed needs and our hardware?
• What data do we need, and are we allowed to train on our calls?
• How do we prove the result is better before a customer hears it?

vakforge answers them from your own data.

The hard part was never the models. It was the decision.

"Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty." Jensen Huang, NVIDIA, on X, 24 July 2026

vakforge is about that last word: owning your voice AI instead of renting it.

How it works

1. init creates a project folder for your language and market.
2. inspect reads your data folder and reports what is in it: languages, personal data, tables that could become tools, audio quality.
3. recommend decides what needs changing. Often the answer is retrieval and tools, with no training at all.
4. prepare, train, eval, serve are built for your project by a coding agent using the vakforge skill. It trains only what recommend asked for, and ships only if eval shows the result beats the base model.

Today the CLI covers init, inspect, recommend, report, validate and the locale packs.

How the decision is made

Every route ends the same way: test it against your own held-out data, and ship only if it beats the base model.

Each goal has its own fix, and most of them are not a full fine-tune. The detailed rules, with how much data each route needs and what hardware it takes, are in docs/DECISION_GUIDE.md.

The problem, for engineers

Open speech-to-speech models exist. Fine-tuning scripts exist for some of them. Evaluation tools and serving frameworks exist. What is missing is one path from "here is what my company knows" to "here is a voice assistant that handles my workflow, I can prove it is better than the base model, and my existing voice client can talk to it without a rewrite." Every team rebuilds that path, and most of them fine-tune when they should have used retrieval.

What's in the repo

1. A small library and command-line tool (pip install vakforge). No machine-learning dependencies, so it runs on any laptop. It holds the dataset format, the data checks, the data inspector, the decision rules and the locale packs.
2. An agent skill (skill/vakforge/). Add it to Claude Code or another coding agent. The agent reads your data, follows the decision rules, and writes the training and serving code for your project. It checks every library it uses against the installed source first, so it does not guess at APIs. It has not been run end to end on a real project yet; that is the next milestone on the roadmap.
3. Recipes (docs/RECIPES.md). Researched paths from an open base model to a running assistant: which model suits which language and GPU, what it can and cannot learn, and the licence to read first. None have been run end to end yet, and the file says so on every page; a recipe is only marked tested once its report is committed.

Use it with a coding agent

The skill is where the knowledge lives. Copy it into your project and the agent follows the vakforge workflow: inspect, recommend, then write and test the prepare, train, eval and serve code for your case, verifying every upstream API against installed source first.

# Claude Code: project-level skill (or ~/.claude/skills/vakforge for every project)
mkdir -p .claude/skills
cp -r path/to/vakforge/skill/vakforge .claude/skills/vakforge

Then ask: "Build a Hinglish support assistant from the data in data/raw." Other agents that read SKILL.md-style instructions work the same way; point them at the folder.

Bring the data you have

inspect reads Markdown, text and HTML documents, and question/answer files (CSV or JSON) as FAQs; CSV, TSV, SQL dumps and JSON/JSONL tables; WhatsApp exports, JSON chat logs, CSV or JSON files with one exchange per row (input/output, instruction/response, customer/agent and similar), and CSV files with one message per row (a role or speaker column and a text or message column); and WAV, FLAC, OGG, Opus (including WhatsApp voice notes) and MP3 audio. Anything else is listed with the reason it was skipped. The table below is where each kind of source ends up on the full path.

Locale packs

The core works in any language. Everything that changes by language or country lives in a locale pack: how money, dates and phone numbers are written, which ID numbers count as personal data, the local privacy and call-recording rules, and which open models support the language. Adding a new market means adding a pack, not changing the core. See docs/LOCALE_PACKS.md, or run vakforge locales.

Recipes

"Duplex" means the assistant can listen while it talks, so callers can interrupt it naturally. Details in docs/RECIPES.md.

Connecting your app

Your assistant runs an open model on your own servers. Nothing calls OpenAI or any other hosted API.

None of this is built yet: serving is written into your project by the agent skill, and this table is the design it follows, not a list of working endpoints.

The intent is that the server speaks the same WebSocket messages as OpenAI's Realtime API, so an app already talking to GPT Realtime changes one URL and clients like Pipecat, LiveKit and Twilio keep working. "Realtime compatible" describes a message format, never a dependency: the model is open and runs on your hardware. More connection types can be added without touching the model:

Why English and Hinglish first

English has the strongest open voice models, so every recipe works for US, UK and Indian English. Hinglish is the hard test: people switch between Hindi and English mid-sentence, write Hindi in Roman or Devanagari script, and say amounts like "2 lakh rupees" on noisy phone lines. If vakforge handles that, adding another language is mostly filling in a new pack.

Quick start

pip install vakforge
vakforge init my-assistant --locale hi-Latn-IN
cd my-assistant
# put your documents, tables, chat exports and call recordings in data/raw/
vakforge inspect data/raw -o runs/inspect.json

inspect prints a summary and writes runs/inspect.json. vakforge recommend runs/inspect.json -o runs/recommend.json reads it and writes the decision.

No data of your own yet? examples/hinglish-shop is a small synthetic project (Hinglish FAQ, two table exports, a WhatsApp chat) with both reports committed next to it, so you can see the output before installing anything.

Documentation

• docs/DECISION_GUIDE.md: what needs customizing, and when not to fine-tune
• docs/GLOSSARY.md: what the terms in inspect and recommend output mean (turns, bars, verdicts, routes)
• docs/LOCALE_PACKS.md: what a locale pack contains, and how to add one
• docs/DATA_FORMAT.md: the dataset format
• docs/ARCHITECTURE.md: how the code fits together, with diagrams
• docs/RECIPES.md: per-model training recipes
• docs/EVALUATION.md: metrics, report format, per-locale benchmarks
• docs/DATA_ETHICS.md: consent, PII, licences, privacy law by region
• docs/ROADMAP.md: status
• CONTRIBUTING.md

Related projects (we build on these, not against them)

Unsloth, LLaMA-Factory, ms-swift, kyutai-labs/moshi-finetune, NVIDIA PersonaPlex, liquid-audio, Qwen-Omni, Pipecat, LiveKit Agents, vLLM-omni, UltraEval-Audio, AI4Bharat, Common Voice

Licence

Apache-2.0 for this code. Each open model keeps its own licence; see docs/RECIPES.md. Datasets you create with vakforge are yours. The consent records vakforge asks for are there to keep it that way.