KonfAI Studio

A single chatbot (à la ChatGPT / Claude, specialized for medical imaging) that drives konfai-mcp end to end. A clinician-researcher points it at their own dataset and, from the conversation alone, onboards data, authors/reuses a model, trains, infers, visualizes results, compares, keeps & reproduces experiments, then deploys the frozen model privately (on-prem or 100% in the browser). KonfAI jobs run on the machine hosting Studio. Chat messages and tool results are sent to the selected language-model endpoint. Use a local endpoint when those exchanges must stay local; app and model downloads may still need network access. See the Studio guide for setup and data handling.

This is a product surface, not a new engine. Every capability maps 1:1 onto an existing konfai-mcp tool. The build is the web UI + a thin bridge (BFF), plus the ONNX export.

See it work

Five real sessions on real data. Each video is one continuous take, sped up but never cut.

Train from your dataset

Point Studio at your images and ask in plain words. Studio inspects the dataset, writes and validates the config, and asks before spending GPU time. Training runs with live curves and validation samples, then the held-out cases are predicted, evaluated, and opened in the viewer next to the ground truth.

https://github.com/user-attachments/assets/541ba05d-a4c0-4b20-b51a-1fa134293858

Iterate and steer, live

Launch one more run and steer it while it trains. Change the learning rate mid-run, stop the job when you decide, evaluate the same held-out cases, and compare every run on a leaderboard that flags optimistic numbers.

https://github.com/user-attachments/assets/af68ab7d-c26f-45b5-9fca-1938058a129c

Use a published model

Pick an app from the zoo and run its published weights on your own cases. GPU inference takes seconds per case, the result is scored against your reference mask, and the segmentation opens over its CT in the viewer.

https://github.com/user-attachments/assets/91310a7f-973e-4098-a23d-ef6cab072c00

Fit a challenge task

How KonfAI is used in challenge season. Studio designs a loss shaped for the task's score, writes it as a real PyTorch module, smoke tests it before any GPU hour, and plugs it into the config by classpath. It trains under a time budget, scores the result the way the challenge does, and claims nothing without a fair baseline.

https://github.com/user-attachments/assets/1c5e299e-8321-43f5-bc0e-fc5d0700bfcd

Prepare a whole dataset

The TRANSFORM workflow processes 75 volumes under a 512 MiB memory cap. The plan states, case by case, what will stream in slabs and what simply fits in memory. The whole dataset goes through in about a minute and comes back verified.

https://github.com/user-attachments/assets/b9b4c01f-7bf0-4f61-86f6-5ae8ce784534

Layout

• konfai_studio/: the Python package (the BFF)
  • server.py: FastAPI that streams the chat over SSE, serves the front, streams volumes to NiiVue
  • Model backends: Claude Code (default), the Claude API or an OpenAI-compatible endpoint
  • web/: the built front (index.html + assets/, git-ignored; logos are committed)
• frontend/: the React + Vite source (chat panel + NiiVue viewer; npm run build emits into web/)
• docs/: the spec and the remote-deployment guide

Run

pip install konfai-studio
konfai-studio                           # -> http://127.0.0.1:8730

The published wheel already ships the built front. The default brain uses your Claude Code subscription (no API key); konfai-studio[openai] and konfai-studio[anthropic] add the alternatives. For a local model: KONFAI_STUDIO_LLM=openai KONFAI_STUDIO_LLM_BASE_URL=http://localhost:11434/v1 KONFAI_STUDIO_MODEL=qwen2.5:14b konfai-studio.

The same choice is in the interface: pick the backend beside the composer and Connect… takes its server URL and key, stored 0600 under the workspaces root. The environment wins on start-up.

Develop from a checkout

The front is git-ignored, so a checkout has to build it. konfai-mcp comes first: from a checkout Studio asks for konfai-mcp at the checkout's last release tag or newer, which the checkout's own satisfies once installed; otherwise pip fetches a release from PyPI. At a release tag the pin is exact.

pip install -e ./konfai-mcp             # must precede studio, see above
pip install -e ./konfai-studio                 # installs the server dependencies
npm --prefix konfai-studio/frontend install    # once
npm --prefix konfai-studio/frontend run build  # builds the front into konfai_studio/web/

Front hot-reload during development: npm --prefix konfai-studio/frontend run dev (proxies to the BFF).

See docs/STUDIO_SPEC.md for the design, and docs/REMOTE.md to serve it beyond loopback.