YOLOZU (萬)

Japanese: Readme_jp.md | Chinese: Readme_zh.md

Company: ToppyMicroServices OÜ | Official page: https://www.toppymicros.com/yolozu/ | PyPI: https://pypi.org/project/yolozu/ | Manual DOI: https://doi.org/10.5281/zenodo.18744926

Evaluate existing predictions

YOLOZU is a commercial product developed by ToppyMicroServices OÜ and provided free of charge. The repository code is licensed under Apache-2.0.

Its stable product lane validates and fairly evaluates existing vision predictions through a stable predictions interface contract.

Give it a wrapped predictions.json, validate the predictions interface contract, and produce a comparable report.

The shortest core-install path is one strict dry-run command:

yolozu eval-coco -d /path/to/dataset -p /path/to/predictions.json --dry-run -o reports/coco_eval.json

For real COCO metrics, install yolozu[coco] and omit --dry-run.

1-Minute Demo

python3 -m pip install -U yolozu
yolozu doctor --proof
yolozu demo instance-seg --run-dir reports/quickstart_instance_seg --progress

Writes reports/quickstart_instance_seg/instance_seg_demo_report.json and visible PNG overlays under reports/quickstart_instance_seg/overlays/. The matching checklist lives at configs/quickstart/instance_seg_demo.yaml. For the full CPU-only DoD path (doctor --proof -> demo -> validate -> eval), see docs/cpu_only_dod.md. If you are unsure what to run next, use the built-in guide:

yolozu guide
yolozu guide --goal first-run
yolozu guide --goal evaluate

Python And AI Quick Use

Use the typed in-process API when another Python program owns the workflow:

from pathlib import Path

from yolozu.api import evaluate_coco

result = evaluate_coco(
    dataset=Path("/absolute/path/to/dataset"),
    predictions=Path("/absolute/path/to/predictions.json"),
    dry_run=True,
)
print(result.to_dict())

Give an AI client the small guaranteed-tool list before exposing wider surfaces:

yolozu-mcp --print-tools --guaranteed --ids-only

See docs/python_api.md and docs/ai_first.md for typed errors, workspace boundaries, MCP setup, and larger opt-in discovery.

Before training, fail closed on an empty or invalid split and ask the train doctor for a machine-readable readiness decision:

yolozu validate dataset /path/to/yolo_dataset --split train --strict
yolozu doctor train-dataset --dataset /path/to/yolo_dataset --split train --output -

For separate COCO annotation and image paths, use --instances together with --images-dir; --dataset is not required. See docs/training_inference_export.md.

flowchart LR
    A["Ultralytics"] --> D["wrapped predictions.json"]
    B["RT-DETR"] --> D
    C["Detectron2 / MMDetection / custom"] --> D
    D --> E["validate"]
    E --> F["evaluate"]
    F --> G["comparable report"]

Read These First

• docs/README.md: top-level docs map and shortest working paths
• docs/predictions_schema.md: the predictions interface contract
• docs/python_api.md: typed in-process validation/evaluation API and error policy
• docs/dataset_processing_matrix.md: dataset source/target, preservation, and qualification boundaries
• docs/bop_tless_protocol.md: Research-stage BOP T-LESS rigid-object 6DoF protocol and evidence boundary
• docs/install.md: install, doctor, and environment setup
• docs/byop_quickstarts.md: checked Ultralytics, Detectron2, MMDetection, and YOLOX export-to-report paths
• docs/case_studies/maskrcnn_eager_torchscript.md: real eager/TorchScript outputs evaluated through one pinned lane
• Searchable web docs: self-contained strict 30-minute path, typed Python API, generated commands and schemas, examples, glossary, and failure guide

Primary Focus

• Stable lane: evaluate precomputed predictions fairly across frameworks and runtimes
• Bridge lane: export or external training flows that emit the same predictions interface contract
• Benchmark lane: qualify backend parity after the stable evaluation path is working
• Research lane: opt-in workflows over already evaluated artifacts

Adaptive Local Vision Roadmap

Environment-aware local image processing remains Experimental delivery work. It does not change the current Stable prediction validation/evaluation surface.

The target design lets an AI client turn natural language into a typed request, then asks YOLOZU to select only among pipelines qualified for the matching task, hardware, runtime, workload, protocol, and license constraints. If the evidence is missing or mismatched, the result is abstention rather than an inferred "best" model. Recommendation and execution remain local and do not implicitly download assets.

Strict request, environment, evidence, eligibility-observation, and SelectionDecision interface contracts are packaged for integration work. yolozu doctor --output - now includes a privacy-safe live environment_profile; failed probes stay unknown and do not prove that an accelerator is absent. The packaged bundle registry now contains three non-promoted Candidate baselines matching the existing model zoo: YOLOX-S, Detectron2 Faster R-CNN R50-FPN 1x, and MMDetection Faster R-CNN R50-FPN 1x. Their pinned weights are fetchable metadata, but their adaptive execution binding is explicitly unbound. Loading and validation do not import a model runtime. Experimental yolozu scout-algorithms now validates one canonical official-source allowlist and, only with --collect, writes a dated candidate inbox. The default command is a network-free, write-free plan. Collected pages are untrusted metadata; raw documents are not retained, and the inbox cannot be loaded as an AlgorithmBundle registry or used as qualification, support, recommendation, adoption, or promotion evidence. Experimental yolozu check-qualification-freshness reads active qualification deadlines and governed runtime/bundle drift without re-running or extending evidence. The repository schedule retains only bounded public identifiers; an explicit site evidence root remains local-only and never uploads site facts. The contract-only OCR result boundary keeps recognized text as inert untrusted user output, separate from detection labels. It ships no OCR model, adapter, document parser, remote service, or support claim. The contract-only streaming boundary defines bounded per-frame results, privacy-safe summaries, unactivated qualification-report record shapes, and explicit selection or abstention. The separate tracking boundary validates session-scoped state transitions against those frame results. This adds no decoder, live runner, tracker, model, or support claim. Candidate screening is implemented as a separate non-executing interface contract. It derives pass, hold, or reject from bounded provenance, integrity, license, local-availability, task/output, runtime, resource, maintenance, security, and human-review facts. Mandatory unknowns remain hold. The sole packaged append-only screening stream contains two current hold decisions from the 2026-08-29 candidate review, and workspace screening input is always operator-asserted. Neither record is a managed pass, so no candidate is made available by this implementation. Explicit workspace catalogs remain operator-asserted and nonselectable. The Experimental yolozu qualify-image-pipeline command now implements pinned no-follow input and asset preflight, a frozen repeat/soak protocol, bounded child-process cancellation, and atomic unactivated qualification_report.json publication. Experimental yolozu activate-qualification-evidence now dry-runs every review, trust, freshness, registry, lifecycle, and stale-head gate before it can append an activation, supersession, or terminal revocation. Mutation requires --approve. Experimental yolozu review-image-pipeline-support-profiles separately reviews one complete ordered exact-measured target set. It reads and, only with --approve, atomically appends to the sole packaged support_profiles.jsonl SSOT. A reviewed set remains dormant: it does not change a lifecycle pointer, activate evidence, bind a runner, download a model, or claim current support. The stream is currently empty. Recommendation and execution use the same loader-derived support-profile provider, and execution reprojects the lifecycle-pinned historical set before any runner session is opened. Experimental yolozu update-image-pipeline-lifecycle adds a separate reviewed, dry-run-first maintenance interface contract for exact disable, enable, license review, terminal global revoke, and explicit per-channel rollback. Mutation requires --approve, the observed lifecycle and support heads, immutable bundle/artifact identities, and an approved public review. A non-none rollback must restore the complete historical advertised profile set and one current repository-managed activation per profile for an eligible same-family target. It cannot choose a newer dormant target, accept a caller-authored subset, use a never-assigned Candidate as a promotion shortcut, promote a bundle, or mutate from metrics alone. The rollback event records the exact historical target-assignment digest. No lifecycle event was appended by this change. Experimental yolozu promote-image-pipeline implements the separate reviewed promotion path. Candidate-to-Experimental and Experimental-to-Stable are distinct operations, and omission of --approve is always a no-write dry-run. The command requires exact source/target pointers and stream heads, the complete canonical ordered support-profile set, and one current repository-managed activation per profile. Stable additionally requires a passed bounded failure-drill report, separate human repository approval, all preregistered absolute gates, and exact zero-tolerance comparison with the current Stable reports when a comparator exists. It cannot use site-managed evidence, infer approval, change the Stable profile set, or modify bundle, support, screening, evidence, report, or artifact inputs. This implementation did not promote any model; the packaged streams remain Candidate-only. Locally emitted reports can reach only site_managed / site_qualified; arbitrary workspace JSON remains nonselectable. Repository-managed trust additionally requires the retained, tracked review workflow and a public review reference. No adaptive runner is bound to those records, so the command currently fails actionably instead of producing dummy evidence. A pure internal selector now applies the fixed trust, compatibility, artifact, evidence, performance, and deterministic ranking rules to already validated in-memory observations. It performs no provider-file, model, runner, or network I/O. The Experimental MCP-only recommend_image_pipeline tool now exposes the same bounded selection policy as a read-only structured recommendation. It validates the typed job and local input, checks non-I/O gates before artifact access, and returns either a complete SelectionDecision or an honest abstention. It does not run inference, download or write assets, parse natural language, or expose absolute paths or raw probe output. Because the three records remain Candidate and the public evidence stream is empty, the default call currently abstains with maturity_disallowed. Experimental MCP-only process_images now accepts that complete selected decision, revalidates the job, current lifecycle/evidence, environment, workload, input, class mapping, and pinned artifact state, and defaults to a no-write dry_run=true. Explicit execution uses only a registered code-owned, network-free route and publishes an atomic managed predictions/provenance/checksum tree. The adaptive runner maps remain empty, so no real adaptive model can currently execute; no model adapter or performance claim was added. An activation record alone does not select or execute a model. Registry loading, an environment profile, a smoke result, and output publication alone are not qualification evidence or proof of human adoption.

The baseline bundle registry report records the current three-way state boundary. The earlier algorithm scout foundation report records the monitored-source, retention, parser, and nonselection boundaries. The earlier candidate screening foundation report records the fail-closed screening and path-derived trust boundaries. The earlier lifecycle maintenance and rollback report records the reviewed mutation, immutable-history, and exact rollback boundaries. The earlier installed-artifact verification report checks the same boundary from source, sdist, wheel, and installed MCP calls. Its positive selector and executor cases use internal fixtures; they do not qualify a real bundle or demonstrate a selected public run.

See the generated roadmap report, the packaged machine-readable projection, and the Beads synchronization rule.

Capability Maturity

• Stable: prediction validation/evaluation, wrapped predictions.json, repo smoke/demo path, install/doctor flow
• Experimental: backend parity, benchmark orchestration, external training handoff, macOS/MPS evaluation paths, TTA
• Research: continual learning, self-distillation, TTT, Hessian refinement, and BOP T-LESS rigid-object 6DoF

These are capability-level boundaries. A Stable parent CLI or manifest entry does not promote opt-in subcommands or flags: export_predictions keeps baseline export Stable, TTA Experimental, and TTT Research.

The BOP lane means rigid-object R,t pose, not human 3D skeleton pose. Its real T-LESS diagnostic has strict GT, three-seed task-native before/after evaluation, and an independent semantic reproduction. The follow-up exports matched pose estimates for the official BOP19 test targets and evaluates them with the pinned official toolkit. The lane remains Research because protocol completion produced only small, seed-inconsistent official and task-native scores; one seed had zero 0.1-diameter pose success. See the diagnostic report and official-test report.

The continual-learning lane now has a one-command, schema-defined three-seed naive-versus-checkpoint-distillation diagnostic: ./.venv/bin/python tools/qualify_sdft_continual.py --output-dir /tmp/yolozu-sdft-qualification. It runs real COCOeval and records baseline-relative FWT, hashes, time, memory, and fairness checks. This is an SDFT-style detector regularizer rather than a faithful reproduction of language-model SDFT, and it remains Research until efficacy is established. The completed 2026-07-28 run is a measured negative result: every real-COCOeval matrix cell and every SDFT-minus-naive delta was zero, so the decision is hold and efficacy is not_established. The hash-verified bundle is available as a GitHub prerelease; the bundle has been independently reproduced in a second Python/Torch environment. See the evidence report. The 2026-07-30 confirmatory spec produced non-zero task scores for all seeds, and an independent run reproduced the protocol and gate outcomes. Two of three seeds passed the preregistered retention/adaptation checks; seed 66 failed the strict old-task improvement gate. Efficacy therefore remains not_established; see the confirmatory report.

Experimental fine-tuning lanes can be audited in one command with ./.venv/bin/python tools/qualify_finetune_lanes.py --output-dir /tmp/yolozu-finetune-qualification. The schema-defined result separates executed training from config projection, records dependency failures and checkpoint/provenance hashes, and keeps the lane Experimental when task-native metrics or non-heuristic labels are absent. The bounded clean-source result remains hold; see the fine-tuning evidence report. The 2026-07-30 follow-up used strict T-LESS GT and executed real training in Ultralytics, HF DETR, and Detectron2 across two environments; five other external runtimes emitted structured availability failures. The runtime evidence remains Experimental and hold. A compatible Linux/CUDA workflow separately completed non-dry training for YOLOX, MMDetection, MMPose, MMSeg, and NVIDIA TAO in two independent runs on the same pinned T4 stack. This establishes compatible-host runtime availability and structural handoff reproducibility, not training quality or checkpoint byte determinism. All five lanes remain Experimental / hold; see the compatible-host report.

TTT comparisons can be run as a fail-closed multi-seed clean/shift matrix with tools/run_ttt_evidence_suite.py; generated metrics do not promote the Research lane. The bounded 2026-07-27 diagnostic bundle is available as a GitHub prerelease; the archive SHA-256 is bb200d0c0a36447f0b6ed262a56ee09bef44ded8f10c55673243080fe1054068. All 30 matrix cells have been independently reproduced with zero semantic differences. This establishes diagnostic reproducibility, not efficacy. See docs/ttt_protocol.md.

A separate 2026-08-01 local diagnostic uses a stronger, fully compatible source checkpoint and a detector-native response objective: it excludes the final no-object class, selects confident foreground queries, and keeps same-query class/box consistency across a weak photometric view. It abstains without backward or optimizer execution and restores normalization buffers when the selected count is below the configured minimum. On the fixed 10-image clean/shifted fixtures, COCO mAP50:95 changed from 0.000990→0.001188 and 0.000330→0.000396, respectively, with no guard stops. This is a positive bounded observation, not independent evidence or an efficacy claim. See the detection-native report.

Production Readiness

• Production-ready today: prediction validation/evaluation and the predictions interface contract
• Needs qualification in your environment: backend parity, benchmark orchestration, SynthGen handoff, macOS/MPS paths
• Research-oriented: continual learning, self-distillation, TTT, Hessian refinement
• Full details: docs/production_readiness.md

Best Fit

• Compare predictions from multiple frameworks or runtimes on the same dataset and pinned evaluation protocol.
• Validate and wrap predictions from your own or a third-party vision stack before running one evaluation path.
• Add CI or regression reports that expose metric, preprocessing, or backend drift.

Not The Best Fit

YOLOZU is not the best fit when you need a managed training platform, hosted inference service, guaranteed support or SLA, or one-click production deployment. If you evaluate only within one framework and do not need a stable cross-stack boundary, that framework's native evaluator may be simpler. Training, benchmark, adapter, and research capabilities are secondary qualified lanes, not the stable product promise.

Why Not Just Use Framework-Native Evaluation?

Framework-native evaluation is convenient inside one stack, but it is harder to compare fairly across stacks. YOLOZU keeps the evaluation boundary at one predictions interface contract so the comparison path stays pinned even when the inference stack changes.

Where To Go Next

• Evaluate precomputed predictions: docs/external_inference.md
• Bring your own model project: docs/byop_quickstarts.md
• Train, export, then evaluate: docs/training_inference_export.md
• YOLO-style and Detectron2 external training lanes (yolozu train --external-backend yolox|detectron2|ultralytics|hf-detr ...): docs/training_inference_export.md
• Current training support matrix and scope boundary: docs/training_inference_export.md#current-training-support
• Training backend interface / capability matrix / orchestration: docs/training_backend_interface.md, docs/training_capability_matrix.md, docs/training_orchestration.md
• Qualify backend-parity and benchmark paths after the main eval lane is working: docs/backend_parity_matrix.md, docs/benchmark_mode.md, docs/benchmark_support_matrix.md
• Inspect a reproducible two-runtime comparison and its committed evidence: docs/case_studies/maskrcnn_eager_torchscript.md
• Qualify a YOLOZU-synthgen handoff in one command: docs/synthgen_repo_integration.md
• Tool and manifest references: docs/tools_index.md, tools/manifest.json

Secondary And Research Lanes

• Training, export, benchmark, SynthGen, and research workflows feed or extend the evaluation boundary.
• External training bridge: YOLOX first, optional Ultralytics and HF DETR bridges second
• SynthGen handoff: docs/synthgen_repo_integration.md
• Research workflows: docs/research_lanes.md
• Real-image showcase: docs/assets/readme_multitask_showcase.png

Repo Users

python3 -m pip install -e .
bash scripts/smoke.sh

More repo-first guidance:

• Docs index: docs/README.md
• Install details: docs/install.md
• Manual sources: manual/README.md

Support, Feedback, And Legal

• Structured support and feedback: docs/support.md
• License policy: docs/license_policy.md
• External training boundary: YOLOX first, optional Ultralytics and HF DETR bridges second
• Apache-2.0 license: LICENSE
• Latest release: GitHub Releases
• Zenodo software DOI: 10.5281/zenodo.18744756
• Zenodo manual DOI: 10.5281/zenodo.18744926