VisionForge

A local-first computer-vision experimentation platform for researchers. Train, validate and compare models on your own GPU — no cloud, no notebooks, no copy-pasted training loops. PyTorch + FastAPI + React in one Python process.

VisionForge replaces ad-hoc Jupyter workflows with a clean, testable, reproducible system where the numbers you report are numbers you can defend: every run records its full provenance, and every comparison can be replicated across seeds with confidence intervals.

Five task families, one interface

Every task panel follows the same canonical layout: experiment name + YAML export/import, a strategy selector, model, training, dataset (with pre-training stats), preprocessing filters and augmentation with live preview.

Built for defensible results

• Multi-seed replicates — train the same config N times under different seeds and report metric = mean ± 95% CI (Student-t) instead of a single point estimate.
• K-fold cross-validation — classification, regression and segmentation; per-fold metrics + mean ± std, with fold-leakage-safe transforms.
• Hyperparameter sweeps — grid, random, or Optuna TPE over any config field by dot-path; one-click architecture-comparison preset.
• Paired significance testing — compare N configurations over the same seeds and get the difference, its bootstrap CI, a paired t or Wilcoxon test (chosen and justified per comparison), Cohen's d_z, and Holm-Bonferroni correction across the family. It refuses to compare runs whose seeds do not line up, and flags when the seed count makes significance unreachable — so "not significant" is never mistaken for "no effect".
• Paper-ready output — every replicates / sweep / K-fold / comparison report is also written as a booktabs LaTeX table, with notes stating what each interval covers and which correction was applied.
• Full provenance — every run writes a versioned run.json with the exact config, seed, per-epoch history, environment (Python, torch/torchvision, numpy, CUDA, cuDNN, GPU model) and a dataset fingerprint, so "same data" is a checkable claim rather than a shared path.
• Reproducibility knobs — seeded runs, optional deterministic cuDNN mode, config schema versioning with migrations, YAML round-trip (export from the GUI, re-run from the CLI).
• Post-training tooling — run history with multi-run comparison and config diff, per-checkpoint testing on new datasets, batch prediction to CSV, Grad-CAM explainability, ONNX export with PyTorch-vs-runtime latency benchmark, TensorBoard scalars per run.
• Dataset utilities — split auto-detection, per-split stats (class balance, image/mask pairing, manifest checks with target distributions), one-shot download from torchvision / Roboflow / Kaggle / Hugging Face.

Installation

Requirements: Python 3.13+. Node.js is only needed to build the frontend from source — the published package already ships the built UI.

On PyPI the distribution is visionforge-studio — the bare visionforge name belongs to an unrelated project. The import name, the CLI command and the project itself are still visionforge.

pip install "visionforge-studio[cu121]"    # NVIDIA CUDA 12.1
visionforge doctor                          # confirms the right wheel for your GPU
visionforge gui

From source

For development, or to run an unreleased commit:

git clone https://github.com/marcus-vreis/VisionForge.git
cd VisionForge
uv venv
# Windows: .venv\Scripts\activate     Linux/macOS: source .venv/bin/activate

PyTorch is intentionally not a plain dependency — its build must match your hardware, and a resolver cannot pick correctly between the CPU and CUDA wheels (ADR-005). You choose one via a hardware extra, and the right index is already wired up for it:

uv pip install -e ".[cu121,dev]"    # NVIDIA CUDA 12.1
# also available: cu118 · cu124 · cu126 · cpu

Not sure which? Ask, and it prints the exact line for your machine:

visionforge doctor

It reads your driver and any torch already installed, so on a machine whose GPU works it says so instead of recommending a downgrade.

Build the web UI once (it is then served by the Python backend — end users never need Node):

cd frontend && npm install && npm run build && cd ..

Check the install actually works before pointing it at your data:

visionforge --version
visionforge selftest --quick     # trains every task on synthetic data, ~15s

Optional extras

uv pip install -e ".[detection,optuna,tensorboard]"

Quickstart

New here? The step-by-step walkthrough — install → built-in dataset download → first run → replicates with confidence intervals → YAML re-run — lives in docs/QUICKSTART.md.

GUI (recommended):

visionforge gui           # opens http://127.0.0.1:8000

Pick a task tab, point the dataset picker at your data (stats render immediately), choose a strategy — single run, K-fold, sweep or replicates — and press Treinar. A live monitor streams epochs; results land in the run history with plots, markdown model cards and artifact paths.

CLI (automation):

visionforge run configs/baseline.yaml        # classification
visionforge run configs/detection.yaml       # any task — dispatched by config

Configs exported from the GUI are the exact wire payload, so they re-run identically from the CLI. All artifacts (checkpoints, plots, run.json, reports) are written under outputs/.

Custom models

Drop a Python file into user_models/ and register it:

from visionforge.models.registry import register_model

@register_model("my_net")
def build_my_net(num_outputs: int) -> nn.Module: ...

Select it via model.custom_model — works for classification, regression and segmentation. See user_models/README.md.

Custom tasks — define a whole new task family (ADR-058)

When your research doesn't fit the five built-in tasks, define your own in one documented Python file — no React, no FastAPI, no training loop:

visionforge new-task cell_counting     # writes user_tasks/cell_counting.py

The generated template trains out of the box on synthetic data. Fill four hooks — build_model, build_loaders, compute_loss, compute_metrics — and a Pydantic Config whose fields become a validated form schema. You get, with zero extra code:

• GET /api/tasks · GET /api/custom/<key>/schema · POST /api/custom/<key>/run (live SSE monitor, TensorBoard, versioned run.json provenance)
• POST /api/custom/<key>/sweep — grid/random/Optuna over any config field, including the ones you declared
• POST /api/custom/<key>/replicates — N seeds → mean ± std ± 95% CI

Training not epoch-shaped (GANs, EM loops)? Override run(cfg, ctx) and own the loop while keeping every contract. A working example ships in user_tasks/example_counting/ (a CNN counting dots in synthetic images — trains in seconds on CPU). Full walkthrough: user_tasks/README.md (PT + EN).

Verifying the install

visionforge doctor checks your environment; visionforge selftest checks the pipeline — it builds tiny synthetic datasets, starts the real API, and trains every task through the same endpoints the browser uses, asserting that each run completes, streams live progress, and stores its report:

visionforge selftest --quick     # one run per task (~15s, CPU, offline)
visionforge selftest             # every task x strategy: simple, K-fold, sweep, replicates, comparison

case                       result    time  detail
classification/replicates  PASS      2.7s  accuracy=1.0000±0.0000
segmentation/cv            PASS      2.0s  miou=0.0783
custom/sweep               PASS      0.6s  best mae=2.4231
regression/comparison      PASS      6.5s  best=baseline 1/1 signif.
...
27/27 cases passed

Filters: --tasks classification,custom, --strategies sweep,replicates, --json out.json. Exit code is non-zero if any case fails, so it drops into CI as-is. It verifies integrity, not model quality — one epoch on synthetic data says nothing about accuracy.

Status

v0.1.0 — first public release. Usable for real work and under active development. Below 1.0 the config schema and HTTP API may change between minor releases; configs carry a schema_version and are migrated on load, so a YAML exported from an older release keeps working.

Verified, not asserted: 1274 backend tests and 102 frontend tests gated in CI, plus a full matrix of 21 (task × strategy) cases trained on real datasets — the corpus, the numbers and the one defect it caught are in documentation/VALIDATION.md.

Known limits worth knowing before you start:

• One training at a time. A second submit gets a 409; an experiment queue is on the roadmap. Run batches from the CLI in the meantime.
• One-click dataset download covers classification only (the torchvision built-ins produce an ImageFolder). Detection, regression, segmentation and anomaly need a dataset already in their layout — see documentation/TRAINING_PLAN.md.
• No K-fold for detection or anomaly, by design: Ultralytics owns its training loop, and an unsupervised validation fold without anomalies measures nothing.
• Windows: keep training.workers at 0–2. Each DataLoader worker is a process that reloads torch's CUDA DLLs, and eight of them exhaust the page file (WinError 1455). The default is already 2 there.
• Dark theme only; a light palette is not designed yet.

Found something? Open an issue — the template asks for visionforge --version and visionforge doctor, which answers most of the questions up front.

Architecture, decisions and contributing

• CHANGELOG.md — what shipped in each release
• documentation/ARCHITECTURE.md — layers, modules, boundaries
• documentation/DECISIONS.md — every architecture decision as an ADR (001–065)
• documentation/TRAINING_PLAN.md — the model × strategy matrix, in four layers of increasing cost
• documentation/VALIDATION.md — the real-dataset validation record
• documentation/CONTRIBUTING.md — dev setup, test/lint gauntlet, PR flow

Backend checks: pytest · ruff check src/ tests/ · mypy src/. Frontend: cd frontend && npx vitest run && npx tsc --noEmit. End-to-end: visionforge selftest (or pytest -m slow for the harness's own live cases — they are deselected from the default run).

Citing

If VisionForge is useful in your research, please cite it — see CITATION.cff (GitHub renders a “Cite this repository” button).

License

MIT