Skip to main content

gemini-video-pipeline

Smart Video Chunker, Adaptive Ingestion Engine & Autonomous Agent Tool for Gemini

gemini-video-pipeline is a production-grade Python library, CLI, and autonomous agent tool engineered to preprocess and chunk long-form videos deterministically, handing prepared, token-budgeted video chunks back to your client applications and Gemini agent workflows.


Features

  • Container & Codec Normalization: Seamlessly normalizes variable frame rate (VFR) and formats (MP4, MOV, MKV, WEBM, AVI) into standard constant-timebase ISO MP4 streams using PyAV without full re-encoding when possible.
  • Narrative Scene Detection: Detects camera cuts via 32-bin HSV color histogram Chi-Square distance (0.35 hard cuts, 0.20 soft cuts).
  • Motion Saliency Filtering: Analyzes 64x64 luminance thumbnail frame differencing to eliminate idle/static camera intervals.
  • Token Budgeting Engine: Enforces Gemini's multimodal video token budget (258 tokens/sec baseline), dynamically throttling FPS (0.2 to 2.0 FPS) and spatial resolution (1080p -> 720p/480p) per chunk.
  • Dual Transport & Ephemeral Staging: For client applications targeting Gemini, automatically formats chunks < 20 MB inline via types.Part.from_bytes() and stages chunks >= 20 MB through the Gemini Files API (client.files.upload()) with active polling and context-managed cleanup.
  • Autonomous Agent Tooling: Native Google ADK 2.x and LangChain structured tools for zero-LLM-cost deterministic video preprocessing that returns chunk manifests directly to your agent.

Installation

pip install gemini-video-pipeline

Option 2: Install Directly from GitHub

pip install "git+https://github.com/delta-fde/pipeline-video-chunker.git"

Option 3: Local Editable Development

git clone https://github.com/delta-fde/pipeline-video-chunker.git
cd pipeline-video-chunker
pip install -e ".[dev]"

Specifying the Chunks Output Directory & Disk Persistence

By default, the library writes chunks to ./chunks and uses context managers to automatically clean up temporary .mp4 chunks upon completion (auto_cleanup=True).

If you want chunk files saved to a custom path and preserved on disk:

1. In Python (PipelineConfig)

Pass output_dir (any relative or absolute path) and set auto_cleanup=False:

from gemini_video_pipeline import VideoStreamPipeline, PipelineConfig

config = PipelineConfig(
    output_dir="/path/to/my_chunks",  # Custom chunks directory (created automatically if missing)
    auto_cleanup=False,               # Set to False to keep chunk files on disk
    target_token_budget_per_chunk=200_000,
    motion_filter_enabled=True,
    motion_threshold=0.10,
)

pipeline = VideoStreamPipeline(config=config)
for chunk in pipeline.process("input.mp4"):
    print(f"Chunk {chunk.chunk_index} saved at: {chunk.file_path}")

2. In Autonomous Agent Tooling (VideoChunkerTool)

Specify output_dir and auto_cleanup during tool initialization:

from gemini_video_pipeline.tools.adk_tool import VideoChunkerTool

tool = VideoChunkerTool(
    output_dir="/path/to/my_chunks",  # Custom destination
    auto_cleanup=False                # Persist chunks on disk
)

# Run tool directly
manifest = tool.run({"video_path": "input.mp4"})
for item in manifest["active_chunks"]:
    print(f"Chunk {item['chunk']}: {item['file_path']}")

3. In the CLI (--output-dir)

Use the --output-dir option (defaults to ./chunks):

gemini-video-pipeline chunk input.mp4 --output-dir /path/to/my_chunks --no-motion-filter

CLI Usage

The CLI command is gemini-video-pipeline:

1. Probe Video Metadata

Inspect container format, codec, dimensions, FPS, VFR status, and duration:

gemini-video-pipeline probe video.mp4

2. Chunk Video onto Disk

Split a video into scene-aligned, token-budgeted MP4 files:

gemini-video-pipeline chunk video.mp4 \
  --output-dir /path/to/my_chunks \
  --budget 200000 \
  --motion-filter \
  --motion-threshold 0.10 \
  --max-duration 300.0

CLI options for chunk:

Option Default Description
--output-dir ./chunks Target folder for generated chunk files
--budget 200000 Maximum token ceiling per video chunk
--motion-filter / --no-motion-filter True Filter out static/idle intervals
--motion-threshold 0.10 Motion saliency cutoff threshold (0.0 to 1.0)
--max-duration 300.0 Maximum duration in seconds for any single chunk

3. Analyze Video Chunks with Gemini

Splits video, uploads chunks, and queries Gemini with temporal grounding metadata (requires GEMINI_API_KEY):

export GEMINI_API_KEY="your-api-key"
gemini-video-pipeline analyze video.mp4 --prompt "Identify any security incidents" --model gemini-2.5-flash

Python API Reference

Pipeline Configuration Options (PipelineConfig)

from gemini_video_pipeline import PipelineConfig

config = PipelineConfig(
    output_dir="./chunks",             # Directory where chunk files are saved
    auto_cleanup=True,                 # Ephemeral deletion of chunk files upon context exit
    target_token_budget_per_chunk=200_000, # Gemini token limit per chunk
    scene_cut_threshold=0.35,          # Chi-Square distance threshold for HARD scene cuts
    soft_cut_threshold=0.20,           # Chi-Square distance threshold for SOFT scene cuts
    motion_filter_enabled=True,        # Whether to skip static/idle footage
    motion_threshold=0.10,             # Mean luminance delta threshold for idle detection
    max_chunk_duration_sec=300.0,      # Max duration before forcing a chunk boundary
    min_chunk_duration_sec=2.0,        # Min chunk duration to prevent micro-fragments
    sample_rate_fps=1.0,               # Base frame sampling rate
    upload_to_files_api=True,          # Route chunks >= 20 MB through Gemini Files API
)

Chunk Object Attributes (VideoChunk)

Each chunk yielded by pipeline.process(video_path) contains:

  • chunk.chunk_index: 1-based index of the chunk.
  • chunk.file_path: Local disk path (inside output_dir).
  • chunk.start_timestamp / chunk.end_timestamp: Global timestamps formatted as HH:MM:SS.mmm.
  • chunk.start_sec / chunk.end_sec: Start and end times in seconds.
  • chunk.duration_sec: Duration of this chunk.
  • chunk.frame_count: Total video frames in the chunk.
  • chunk.source_fps: Source frame rate.
  • chunk.average_motion_score: Average motion saliency score (0.0 to 1.0).
  • chunk.estimated_tokens: Estimated Gemini multimodal token consumption.
  • chunk.sampling_plan: Token budget plan with target_fps and optional scale_resolution.
  • chunk.to_gemini_content(): Converts the chunk directly into a types.Part (inline bytes or Files API URI).

Autonomous Agent Integration

Google ADK 2.x

from gemini_video_pipeline.tools.adk_tool import VideoChunkerTool

tool = VideoChunkerTool(output_dir="./agent_chunks", auto_cleanup=False)

# Register as function declaration for Gemini function calling:
func_declaration = tool.as_function_declaration()

# Or pass directly as an ADK callable:
agent_tool = tool.as_adk_callable()
manifest = agent_tool(
    video_path="security_cam.mp4",
    token_budget_per_chunk=150_000,
    focus_high_motion_only=True
)

LangChain

from gemini_video_pipeline.tools.langchain_tool import create_langchain_video_chunker_tool

tool = create_langchain_video_chunker_tool(output_dir="./langchain_chunks")

Testing & Verification

  1. Run Unit and Integration Tests:

    pytest tests/ -v --cov=gemini_video_pipeline --cov-report=term-missing
    
  2. Run Linting and Static Type Checking:

    flake8 src tests --max-line-length=120
    mypy src
    
  3. Run Self-Contained Examples:

    python examples/01_direct_library_usage.py
    python examples/02_motion_filtered_audit.py
    python examples/03_adk_agent_workflow.py
    
  4. Test Across All Formats, Codecs & Lengths (including >10min and ~30min):

    # Generate standard multi-format test videos (MP4, MOV, MKV, WEBM, AVI, 3s to 3min, VFR, CCTV)
    python scripts/generate_test_videos.py
    
    # Generate long test videos (12-minute multi-scene video and 30-minute narrative video)
    python scripts/generate_long_videos.py
    
    # Process and verify all 12 test videos with output chunks on disk
    python scripts/test_all_videos.py
    
  5. Test Variable Token Budgets & Chunk Duration Constraints: Run the verification matrix across a spectrum of token budgets (5k to 500k tokens) and chunk duration limits (15s to 240s):

    python scripts/test_variable_budgets_and_chunks.py
    

Metadata

Release files for gemini-video-pipeline 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gemini-video-pipeline 0.1.0
File Size Uploaded Provenance Actions
gemini_video_pipeline-0.1.0.tar.gz 31.1 kB Without provenance Details for gemini_video_pipeline-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for gemini-video-pipeline 0.1.0
File Interpreter ABI Platform Actions
gemini_video_pipeline-0.1.0-py3-none-any.whl Python 3 none any Details for gemini_video_pipeline-0.1.0-py3-none-any.whl

Total release size: 71.0 kB

Release files / gemini_video_pipeline-0.1.0.tar.gz

Download URL gemini_video_pipeline-0.1.0.tar.gz
Size 31.1 kB
Tags Source
SHA-256 checksum
How to use checksums
57bef8ea3c7fd31ca02bb725736fdb770585d9733f6f82d3f14e9d58d9a3e972
BLAKE2b-256 checksum
How to use checksums
672639d1826676404adcc6a652b257b85acac6005aad888ea31f6827ae69c16c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.5

Release files / gemini_video_pipeline-0.1.0-py3-none-any.whl

Download URL gemini_video_pipeline-0.1.0-py3-none-any.whl
Size 39.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
258f72eebbee7d1502d1b0fb856cc4dfdf1cb6cb0637dcb63d4aab522e6d4d9b
BLAKE2b-256 checksum
How to use checksums
6d084a701caa292e8f5f10fd8e089f9ba988157c616b91aeabd18c131bd4d18a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.5

Security for gemini-video-pipeline 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page