VidChain is a local-first multimodal RAG framework powered by the IRIS Engine (Intelligent Retrieval & Insight System). It decomposes video into visual, auditory, OCR, and temporal signal streams and fuses them into a queryable intelligence layer, intended for forensic analysis, security auditing, and automated video summarization with on-device privacy by default.

Table of Contents

1. Overview
2. Features
3. Installation
4. Configuration
5. Quick Start
6. CLI Reference
7. SDK: Modular Sensor Matrix
8. REST API
9. Architecture
10. Troubleshooting
11. Contributing
12. License

Overview

VidChain turns raw video into a queryable intelligence layer. Each ingested video is processed through a modular sensor pipeline (visual, audio, OCR, motion, behavioral), decomposed into an isolated Temporal Knowledge Graph, and fused with vector retrieval to produce grounded, timestamp-cited answers. Inference runs entirely on-device by default; cloud models are supported as an explicit opt-in, not a requirement.

Features

Installation

Prerequisites

Option A — Install from PyPI (recommended)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install vidchain

Option B — Install from source (for development)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
git clone https://github.com/rahulsiiitm/videochain-python
cd videochain-python
pip install -e .

Model weights (required either way)

ollama pull moondream   # Vision Language Model
ollama pull llama3      # Language Model for reasoning & routing

CPU Fallback: If no CUDA device is detected, VidChain automatically degrades to CPU mode — no code changes required.

Configuration

No other environment variables are required for the default local-only configuration. Any LiteLLM-compatible provider can be substituted for --llm or --vlm; check the LiteLLM provider docs for the corresponding key name if using a provider other than Gemini or Ollama.

Quick Start

from vidchain import VidChain

vc = VidChain(db_path="./forensic_vault")

# Ingest video (runs full default pipeline)
video_id = vc.ingest(video_source="interview_01.mp4")

# Query
response = vc.ask("What is the main topic of discussion?", video_id=video_id)
print(response)

# Summarize
summary = vc.summarize_video(video_id=video_id, mode="concise")
print(summary)

CLI Reference

vidchain-serve

Launches the FastAPI backend and Next.js dashboard.

vidchain-serve

• API available at http://localhost:8000
• Dashboard opens at http://localhost:8000

vidchain-analyze

Headless video ingestion from the terminal.

vidchain-analyze path/to/video.mp4 --vlm moondream

Model substitution: VidChain uses LiteLLM, so any compatible model can be swapped in, including cloud models if higher reasoning quality is preferred over on-device execution:

# Local (default)
vidchain-analyze video.mp4 --llm "ollama/llama3"

# Cloud (opt-in, requires API key export)
export GEMINI_API_KEY="your_api_key"
vidchain-analyze video.mp4 --llm "gemini/gemini-2.5-flash"

# Custom VLM
vidchain-analyze video.mp4 --vlm "llava:7b"

SDK: Modular Sensor Matrix

VidChain uses a LangChain-inspired composable pipeline. Each Node handles one sensing modality; chains are assembled per use case.

Available Nodes

Custom Pipeline Example

from vidchain import VidChain
from vidchain.pipeline import VideoChain
from vidchain.nodes import AdaptiveKeyframeNode, LlavaNode, OcrNode, TrackerNode, YoloNode

vc = VidChain(db_path="./forensic_vault")

surveillance_chain = VideoChain(nodes=[
    AdaptiveKeyframeNode(change_threshold=1.5),  # High sensitivity
    YoloNode(),                                  # Structured detections
    TrackerNode(),
    LlavaNode(model_name="moondream"),           # Semantic scene description
    OcrNode(),
])

video_id = vc.ingest(
    video_source="gate_camera_04.mp4",
    chain=surveillance_chain
)

response = vc.ask(
    "Were there any vehicles with visible license plates after 14:00?",
    video_id=video_id
)
print(response)

REST API

Exposed when running vidchain-serve.

Architecture

Isolated GraphRAG

Each ingested video generates a dedicated Temporal Knowledge Graph (.pkl). The RAG engine retrieves semantically relevant chunks from ChromaDB and fuses them with structured graph data (co-occurrences, tracking IDs, timestamps). Memory boundaries are strictly enforced — no cross-video context bleed.

The Neural Lens

Every query response is paired with a Base64-encoded visual snapshot extracted directly from the referenced timestamp, providing visual grounding for AI-generated claims.

Troubleshooting

Contributing

Issues and pull requests are welcome via GitHub Issues. For substantial changes, open an issue first to discuss scope before submitting a PR.

License

MIT — See LICENSE for details.

Author: Rahul Sharma — IIIT Manipur Portfolio · GitHub