cineinfini-audit 1.0.0

CineInfini: Adaptive Multi-Stage Video Quality Audit Pipeline

CineInfini — Video Quality Audit Pipeline

A no-reference, modular, per-shot video quality auditor for natural and AI-generated video. Pure-CV first; ML-based metrics are opt-in and gracefully degrade when their weights aren't installed.

The intended audience is research and rigorous post-production work, not consumer dashboards. The repository ships:

• 21 quality modules (16 pure-CV + 5 ML-required, all real, no stubs);
• A reproducibility layer (seeded RNGs, environment capture, content hashes, experiment manifests);
• A statistical evaluation layer (Pearson / Spearman / Kendall with bootstrap CIs, VQEG-standard 4-parameter logistic mapping, Steiger Z paired-correlation test, RMSE / MAE);
• Calibration results on three real datasets (n=862 across KoNViD-1k, T2VQA-DB, VideoFeedback) with Tier A/B/C honest-claims hierarchy;
• Phase-4 verdict aggregation (ACCEPT / REVIEW / REJECT) with explicit per-shot gates;
• 16 publication-ready figures generated from those results;
• A working paper draft in docs/PAPER_DRAFT.md.

Quickstart

git clone <repo>
cd cineinfini
pip install -e .

# Sanity check: verify the new statistical / repro layer
pytest tests/test_repro.py tests/test_eval.py tests/test_perf.py -q

# Run a benchmark on your own predictions vs MOS
cineinfini benchmark predictions.csv -o out/

# Print the environment fingerprint that experiments would record
cineinfini repro-info

Honest scope

CineInfini does:

• Compute per-shot quality metrics on a video, with explicit gates, reproducibly.
• Quantify a metric's MOS correlation with bootstrap-stabilized 95% CIs, the right way to put error bars on rank correlations.
• Compare two metrics' correlations on the same dataset using Steiger's Z, the right test for paired correlations sharing a common variable.
• Record an experiment manifest (config + env + data hashes + seed + metrics + artifact hashes) sufficient to reproduce a run.

CineInfini does not claim:

• "State of the art" anywhere. Our calibration results on KoNViD, T2VQA-DB and VideoFeedback are honest pure-CV numbers — strong for pure-CV (Spearman ρ = 0.566 on KoNViD with Random Forest, n=392), not competitive with ML-trained methods like DOVER or FAST-VQA on natural video.
• "32 modules", "50 modules", "human-aligned at ρ = 0.88". The repo contains 21 real, tested modules. We do not pad the count.
• Production-grade SaaS deployment. The code is reproducible and benchmarkable; making it production-grade in any external system is a deployment claim that requires deployment evidence.
• A unique combinatorial-optimization "law". The MILP / Tabu Search result is a stable empirical regime of super-linear cost amplification under coupled decision constraints — Tier A in the paper draft. The natural-coupling-strength claim is explicitly Tier C (does not survive validation as a graph-construction invariant). Read docs/PAPER_DRAFT.md § "Claims hierarchy".

What you get from one audit run

A single cineinfini audit produces:

Format	What
data.json	Per-shot gates + ACCEPT/REVIEW/REJECT verdicts
dashboard.html / .pdf / .md	Human-readable reports
*.vbench.json	16-dim VBench-compatible export
5-axis VideoScore + composite	Visual / Temporal / Dynamic / Alignment / Factual
dover_aesthetic, dover_technical	DOVER scores when DOVER weights are installed
fastvqa_score	FAST-VQA score when FAST-VQA weights are installed

When DOVER / FAST-VQA weights are missing, the corresponding modules report available: false rather than fabricating a number. This is the project's no-fake-science rule.

Reproducibility layer (v1.0.0 addition)

Every benchmark run can record a self-contained manifest:

from cineinfini.repro import ExperimentTracker
from cineinfini.eval import evaluate_predictions
import numpy as np

predictions = np.loadtxt("predictions.txt")
mos = np.loadtxt("mos.txt")

with ExperimentTracker(name="my_metric_vs_konvid",
                       output_dir="./out") as tr:
    tr.set_seed(42)
    tr.record_dataset("KoNViD-1k", path="/data/konvid", n_samples=1200)
    report = evaluate_predictions(
        predictions, mos,
        n_bootstrap=1000, confidence_level=0.95, seed=42,
    )
    tr.record_metrics(report.to_dict())

The resulting manifest_<uuid>.json contains:

• A snapshot of the config dict you passed in.
• The Python version, OS, tracked dependency versions, CUDA device info if torch is installed, and the git commit + dirty flag.
• SHA-256 of the predictions CSV and a Merkle hash of the dataset directory (with optional glob filters).
• The seed and the report of which RNGs were actually pinned.
• The full evaluation report with bootstrap CIs.
• Any artifacts written by the experiment, with content hashes.

This is the contract: if the manifest exists and the data is accessible, the run can be reconstructed. No remote logging backend, no proprietary format.

Statistical evaluation layer (v1.0.0 addition)

from cineinfini.eval import (
    spearman, plcc, fit_logistic_4p, apply_logistic_4p, steiger_z,
    evaluate_predictions,
)

# Spearman with bootstrap 95% CI (1000 resamples, seed-reproducible)
res = spearman(predictions, mos, n_bootstrap=1000, seed=42)
print(res)
# spearman=0.5660 [0.4910, 0.6328] (n=392, p=2.34e-32)

# VQEG protocol: logistic-fit predictions before computing PLCC
fit = fit_logistic_4p(predictions, mos)
fitted = apply_logistic_4p(fit, predictions)
print(plcc(fitted, mos, n_bootstrap=1000))

# "Is metric A significantly better than metric B at predicting MOS?"
# The right test is Steiger Z, not CI overlap:
sig = steiger_z(metric_a, metric_b, mos)
print(sig)

# One-shot: SRCC + KRCC + PLCC-after-fit + RMSE-after-fit, all with CIs
print(evaluate_predictions(predictions, mos, n_bootstrap=1000, seed=42))

All four kinds of CIs (and the logistic fit) are JSON-serializable and drop directly into an ExperimentTracker manifest.

Existing scientific results — now with bootstrap CIs

These numbers are reproduced from the per-video data shipped in docs/figures/ via the v1.0.0 statistical layer (1000-sample bootstrap, seed=42). The script is scripts/v1_bootstrap_all_datasets.py and writes both a JSON manifest (with full env + data hashes) and a paper-ready Markdown table to docs/figures/v1_bootstrap/.

Dataset	n	d	SRCC (95% CI)	PLCC after fit (95% CI)	Steiger Z vs Ridge
KoNViD-1k (natural)	392	16	+0.563 [+0.483, +0.629]	+0.582 [+0.508, +0.650]	Z=2.131, p=0.033
T2VQA-DB (AIGC)	422	16	+0.443 [+0.369, +0.515]	+0.452 [+0.371, +0.526]	Z=3.208, p=0.0013
VideoFeedback (AIGC)	48	16	+0.367 [+0.078, +0.600]	+0.390 [+0.132, +0.603]	Z=1.701, p=0.089

Random Forest hyperparameter sensitivity (item §3.2 in docs/REVIEW_ANALYSIS.md): the headline SRCC swings less than 0.006 on KoNViD and T2VQA across n_estimators ∈ {100, 200, 500} and max_depth ∈ {None, 8}. The paper baseline (n_est=200) is robust.

Reproduce in 2 minutes:

python scripts/v1_bootstrap_all_datasets.py
cat docs/figures/v1_bootstrap/v1_bootstrap_summary.md

The regime-aware sign-flip analysis is in scripts/regime_aware_regression.py and docs/PAPER_DRAFT.md § 4.

Project structure

src/cineinfini/
├── core/        # config, registry, bootstrap, metric kernels
├── modules/     # 21 quality modules (each @register_module)
├── pipeline/    # orchestrator + audit driver
├── io/          # readers, exporters, renderers (md / html / pdf / vbench)
├── aggregators/ # 5-axis VideoScore-style fusion
├── metrics/     # advanced pure-CV NSS / LBP / HOG / DCT / etc.
├── repro/       # v1.0.0 — seeds, env, hashing, ExperimentTracker
├── eval/        # v1.0.0 — Pearson/Spearman/Kendall + bootstrap, PLCC,
│                #          Steiger Z, RMSE, evaluate_predictions
├── perf/        # v1.0.0 — query_devices, ParallelMap, Profiler
├── cli/         # click CLI: audit, score, benchmark, repro-info, ...
└── types/       # shared dataclasses

Tests

pytest tests/ -q          # 339 passing in ~46s on 2 vCPU, no GPU
pytest tests/test_repro.py tests/test_eval.py tests/test_perf.py -v

Documentation

Doc	What
docs/PAPER_DRAFT.md	Full paper draft with Tier A/B/C claims hierarchy
docs/REVIEW_ANALYSIS.md	v1.0.0 — honest analysis of paper-draft strengths and reviewer risks
docs/USER_MANUAL.md	All CLI commands, modules, profiles, outputs
docs/STATUS.md	Exhaustive: done / stubbed / gated / not done
docs/MISSING_ASSETS.md	What weights / datasets are needed and how to get them
docs/QUICKSTART.md	5-minute tour
notebooks/	Runnable Jupyter notebooks

Citation

@software{cineinfini,
  author  = {Benbrahim, Salah-Eddine},
  title   = {CineInfini: Adaptive Multi-Stage Video Quality Audit Pipeline},
  year    = {2026},
  version = {1.0.0},
  license = {MIT},
}

License

MIT — see LICENSE.