VisionOps-AI

visionops is a CLI-first application for pre-deployment reliability testing and root-cause analysis (RCA) of computer-vision pipelines and repositories.

It is designed to answer not only whether performance drops under corruption, but also where the failure likely starts in the pipeline and what to fix next.

What it does

• Runs a baseline pipeline in a sandbox-style workspace.
• Generates corruption suites (blur, occlusion, low light, compression, crop truncation, perspective warp, noise).
• Re-runs pipeline and metrics on corrupted datasets.
• Compares baseline vs suite metrics and detects regressions.
• Performs stage localization using runtime evidence (metrics, logs, execution behavior).
• Uses AI agents for repo mapping, RCA, and fix recommendations.
• Writes summary.json, artifact_manifest.json, report.md, and report.html.

A key differentiator in Visionops is repository understanding. Before generating RCA hypotheses, the system builds a lightweight structural map of the target codebase (entry points, pipeline stages, likely data flow, and high-signal files) and combines that map with runtime evidence from regressions, logs, and command outcomes. This repo-aware context helps it produce recommendations tied to real files and likely failure boundaries instead of generic “check your model” advice. In practice, that means faster triage, more actionable fixes, and better trust in the output because each hypothesis is grounded in both observed behavior and code structure.

Install

python -m pip install visionops

Environment Variables

The CLI loads .env automatically at startup (without overriding already-set shell variables).

Optional keys:

• GEMINI_API_KEY
• GITHUB_APP_PRIVATE_KEY
• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_DEFAULT_REGION

If GEMINI_API_KEY is missing, agent behavior falls back to deterministic logic.

CLI commands

1) Create starter config

visionops init-config --path config.yaml

2) Autogenerate config

visionops autoconfig --repo-path /path/to/cv-repo --output config.yaml

This scans a repository and infers:

• install command
• baseline run command
• metrics command
• dataset and ground-truth paths
• task type and suite defaults

Review the generated file before running.

3) Validate config

visionops validate-config --config config.yaml

To preview without writing and inspect confidence metadata:

visionops autoconfig --repo-path /path/to/cv-repo --dry-run --explain

4) Run full local prototype (primary path)

visionops local-run --repo-path /path/to/cv-repo --config config.yaml

This performs:

1. sandbox copy of repo
2. install command
3. baseline run + metrics
4. corruption dataset generation
5. per-suite reruns + metrics
6. regression analysis
7. stage localization
8. repo mapping + RCA + fix suggestions
9. report generation

5) Run AWS path (secondary path)

visionops aws-run --repo owner/repo --config config.yaml

Behavior:

• checks AWS credentials/config
• starts CodeBuild build
• polls status until terminal state
• returns build id, status, logs, artifact location
• private repo support uses GitHub App auth (no PAT)

If setup is incomplete, command exits gracefully with actionable guidance.

6) Pretty print an existing report

visionops report --path ./visionguard_output_YYYYMMDDTHHMMSSZ

7) Environment health check

visionops doctor --config config.yaml --repo-path /path/to/cv-repo

This validates config, checks key environment variables, and runs autoconfig sanity checks.

8) Built-in metrics fallback

If a target repo does not provide a metrics script, you can use built-in YOLO-format scoring:

visionops builtin-metrics --pred /path/to/pred --gt /path/to/gt

AI-assisted RCA approach

Three AI agents:

• RepoMapperAgent: scans repo and identifies likely pipeline files/stages.
• RCAAgent: combines runtime evidence + repo map into ranked hypotheses.
• FixAgent: converts rule-based remediations into concrete fix suggestions.

By default the application uses Gemini 2.5 Pro via OpenAI-compatible API shape. Set GEMINI_API_KEY in your environment or in .env (see above). If it is missing, all agents use deterministic fallback logic.

Evidence grounding

RCA is grounded in:

• metric deltas
• per-suite regressions
• logs and return codes
• stage localization heuristics
• repo-aware file mapping

The LLM is never the sole source of truth.

AWS sandbox expectations

Before aws-run:

1. configure AWS credentials (aws configure or env vars)
2. create a CodeBuild project named in config (aws.codebuild_project)
3. ensure IAM allows CodeBuild + S3 artifact access
4. optionally configure GitHub App values for private repos

Project Links

• Homepage: https://visionops-ai.web.app/

License

Proprietary. Copyright (c) 2026 VisionOps AI Lab. All rights reserved.