Shot Detection Python

Standalone Python package for shot boundary detection.

It takes a video file, runs TransNetV2-style ONNX inference on low-resolution RGB frames, and returns shot ranges with millisecond timestamps.

Built by Seeknetic. If you want to make video shots searchable and enable more professional video understanding workflows, visit seeknetic.com.

What it does

• Probes the input video with ffprobe
• Extracts low-resolution frames with ffmpeg
• Runs sliding-window ONNX inference
• Finds shot boundaries
• Converts them into {start_ms, end_ms} shot segments

Installation

pip install shot-detection

Import from shot_detection:

from shot_detection import ShotDetector

You also need:

• ffmpeg
• ffprobe

On first run, the package downloads the default model automatically:

• URL: https://download.shotai.io/model/shot-detection/transnetv2_open_fp16.onnx
• cache dir:
  • Linux/macOS: ~/.cache/shot-detection/models/
  • Windows: %LOCALAPPDATA%\\shot-detection\\models\\

You can override the cache root with SHOT_DETECTION_CACHE_DIR.

Usage

from shot_detection import ShotDetector

detector = ShotDetector()
shots = detector.detect("/path/to/video.mp4")

for shot in shots:
    print(shot.start_ms, shot.end_ms)

Advanced usage

from shot_detection import detect_shots

shots = detect_shots(
    video_path="/path/to/video.mp4",
    threshold=0.5,
    min_shot_duration_ms=500,
)

Custom model path

from shot_detection import ShotDetector

detector = ShotDetector(model_path="/path/to/custom-transnetv2.onnx")

Notes

• The package expects the ONNX model input to accept 100-frame windows at 48x27 RGB.
• ffmpeg decode is adaptive: it prefers system-native hardware decoding when available and falls back to software decoding automatically.
• CUDA is intentionally not part of the default decode plan.

Integration with Seeknetic SDK

If you want to run shot-level embedding or tagging jobs after boundary detection, you can combine this package with the Seeknetic Python SDK.

For the full workflow, see docs/seeknetic-sdk-integration.md.

Install the SDK separately:

pip install seeknetic

Set SEEKNETIC_API_KEY before calling the SDK:

export SEEKNETIC_API_KEY="your_api_key"

A paid Seeknetic account and API access are required. You can get an API key from accounts.seeknetic.com.

Quick example

import os
import uuid

from seeknetic import Seeknetic
from shot_detection import ShotDetector

video_path = "/path/to/video.mp4"
shots = ShotDetector().detect(video_path)
client = Seeknetic(api_key=os.environ["SEEKNETIC_API_KEY"])
submitted_requests = {}

for shot in shots:
    request_id = str(uuid.uuid4())
    submitted_requests[shot.index] = {
        "request_id": request_id,
        "start_ms": shot.start_ms,
        "end_ms": shot.end_ms,
    }

    upload = client.preprocess_and_upload(
        video_path=video_path,
        service="embedding",
        request_id=request_id,
        start_ms=shot.start_ms,
        end_ms=shot.end_ms,
    )

    client.video_embedding.encode_video.submit_async(
        tensor={
            "video_input_key": upload.r2_keys["video_input"],
            "audio_input_key": upload.r2_keys["audio_input"],
        },
        request_id=upload.request_id,
    )

print(submitted_requests)

For tagging, switch service="embedding" to service="tagging" and use client.video_tagging.submit_async_job(...).