VideoQL

Searching through video data by asking the right questions.

Install it from PyPI

pip install video_ql

Usage

video_ql provides both a Python API and a CLI interface for video analysis and querying.

Python API

from video_ql import VideoQL
from video_ql.models import Query, QueryCondition, OrCondition, QueryConfig

# Define your queries
queries = [
    Query(
        query="Is the driver present in the forklift?",
        options=["yes", "no"]
    ),
    Query(
        query="Where is the forklift currently at?",
        options=["Truck", "Warehouse", "Charging"]
    ),
    Query(
        query="Is the forklift currently carrying cargo?",
        options=["yes", "no"]
    )
]

# Initialize VideoQL
video_ql = VideoQL(
    video_path="path/to/your/video.mp4",
    queries=queries,
    context="You are viewing the POV from inside a forklift"
)

# Analyze entire video
results = video_ql.analyze_video(display=True)

# Query specific conditions using our Pydantic models
query_config = QueryConfig(
    queries=[
        QueryCondition(
            query="Is the driver present in the forklift?",
            options=["yes"]
        )
    ]
)

# Query the video
matching_frames = video_ql.query_video(query_config)

You can also have VideoQL automatically generate the queries and query config as shown below:

from video_ql import VideoQL
from video_ql.query_proposer import generate_queries_from_context

# Define the context for your video analysis
context = "You are watching a construction site for safety compliance monitoring."

# Automatically generate queries based on the context using the selected model
queries = generate_queries_from_context(
    context=context,
    model_name="gpt-4o-mini",  # Model can be substituted as desired
    num_queries=5
)

# Initialize VideoQL with generated queries
video_ql = VideoQL(
    video_path="path/to/your/video.mp4",
    queries=queries,
    context=context
)

# Proceed with video analysis as usual
results = video_ql.analyze_video(display=True)

Command Line Interface

Natural Language Analysis

Use the CLI tool to analyze your video

$video_ql --video path/to/video.mp4
=====================================
   Welcome to Interactive VideoQL   
=====================================

First, let's create a context for your video analysis.
Describe the video content and what you're interested in tracking or analyzing.
Video context:  ... your context here

Generating queries based on your description...

Generated queries:
1. Query 1?
   Options: Yes, No
2. Another generated query?
   Options: ...

... enjoy your video analysis

YAML Analysis

1. Create a config file (config.yaml):

queries:
  - query: "Is the driver present in the forklift?"
    options: ["yes", "no"]
  - query: "Where is the forklift currently at?"
    options: ["Truck", "Warehouse", "Charging"]
  - query: "Is the forklift currently carrying cargo?"
    options: ["yes", "no"]
context: "You are viewing the POV from inside a forklift"
fps: 1.0
tile_frames: [3, 3]
frame_stride: 9
max_resolution: [640, 360]

2. Create a query file (query.yaml):

queries:
  - OR:
    - query: "Is the driver present in the forklift?"
      options: ["yes"]

3. Run the CLI:

python3 -m video_ql.yaml_analysis --video path/to/video.mp4 \
         --config config.yaml \
         --query query.yaml \
         --output results/query_results \
         --threads 100 \
         --display

You may also process a single frame using the following

python3 -m video_ql.single_frame \
         --image path/to/image.png \
         --config config.yaml \
         --output results/query_results.json \
         --display

The query proposer helps you automatically generate relevant queries for your video content based on a provided context.

from video_ql.query_proposer import generate_queries_from_context, save_queries_to_yaml

# Generate queries based on context
context = "Security camera footage of a parking lot at night"
queries = generate_queries_from_context(
    context=context,
    model_name="gpt-4o-mini",  # or "claude-3-haiku-20240307" 
    num_queries=5
)

# Save queries to a YAML file
save_queries_to_yaml(queries, "generated_queries.yaml")

You can use the query proposer from the command line:

python -m video_ql.query_proposer \
    --context "Dashcam footage of urban driving in rainy conditions" \
    --model "gpt-4o-mini" \
    --num-queries 7 \
    --output "dashcam_queries.yaml"

Development

Read the CONTRIBUTING.md file.