gemini-video-pipeline
Smart Video Chunker, Adaptive Ingestion Engine & Autonomous Agent Tool for Gemini
gemini-video-pipeline is a production-grade Python library, CLI, and autonomous agent tool engineered to preprocess and chunk long-form videos deterministically, handing prepared, token-budgeted video chunks back to your client applications and Gemini agent workflows.
Features
- Container & Codec Normalization: Seamlessly normalizes variable frame rate (VFR) and formats (MP4, MOV, MKV, WEBM, AVI) into standard constant-timebase ISO MP4 streams using PyAV without full re-encoding when possible.
- Narrative Scene Detection: Detects camera cuts via 32-bin HSV color histogram Chi-Square distance (
0.35hard cuts,0.20soft cuts). - Motion Saliency Filtering: Analyzes 64x64 luminance thumbnail frame differencing to eliminate idle/static camera intervals.
- Token Budgeting Engine: Enforces Gemini's multimodal video token budget (258 tokens/sec baseline), dynamically throttling FPS (0.2 to 2.0 FPS) and spatial resolution (1080p -> 720p/480p) per chunk.
- Dual Transport & Ephemeral Staging: For client applications targeting Gemini, automatically formats chunks
< 20 MBinline viatypes.Part.from_bytes()and stages chunks>= 20 MBthrough the Gemini Files API (client.files.upload()) with active polling and context-managed cleanup. - Autonomous Agent Tooling: Native Google ADK 2.x and LangChain structured tools for zero-LLM-cost deterministic video preprocessing that returns chunk manifests directly to your agent.
Installation
Option 1: Install from PyPI (Recommended)
pip install gemini-video-pipeline
Option 2: Install Directly from GitHub
pip install "git+https://github.com/delta-fde/pipeline-video-chunker.git"
Option 3: Local Editable Development
git clone https://github.com/delta-fde/pipeline-video-chunker.git
cd pipeline-video-chunker
pip install -e ".[dev]"
Specifying the Chunks Output Directory & Disk Persistence
By default, the library writes chunks to ./chunks and uses context managers to automatically clean up temporary .mp4 chunks upon completion (auto_cleanup=True).
If you want chunk files saved to a custom path and preserved on disk:
1. In Python (PipelineConfig)
Pass output_dir (any relative or absolute path) and set auto_cleanup=False:
from gemini_video_pipeline import VideoStreamPipeline, PipelineConfig
config = PipelineConfig(
output_dir="/path/to/my_chunks", # Custom chunks directory (created automatically if missing)
auto_cleanup=False, # Set to False to keep chunk files on disk
target_token_budget_per_chunk=200_000,
motion_filter_enabled=True,
motion_threshold=0.10,
)
pipeline = VideoStreamPipeline(config=config)
for chunk in pipeline.process("input.mp4"):
print(f"Chunk {chunk.chunk_index} saved at: {chunk.file_path}")
2. In Autonomous Agent Tooling (VideoChunkerTool)
Specify output_dir and auto_cleanup during tool initialization:
from gemini_video_pipeline.tools.adk_tool import VideoChunkerTool
tool = VideoChunkerTool(
output_dir="/path/to/my_chunks", # Custom destination
auto_cleanup=False # Persist chunks on disk
)
# Run tool directly
manifest = tool.run({"video_path": "input.mp4"})
for item in manifest["active_chunks"]:
print(f"Chunk {item['chunk']}: {item['file_path']}")
3. In the CLI (--output-dir)
Use the --output-dir option (defaults to ./chunks):
gemini-video-pipeline chunk input.mp4 --output-dir /path/to/my_chunks --no-motion-filter
CLI Usage
The CLI command is gemini-video-pipeline:
1. Probe Video Metadata
Inspect container format, codec, dimensions, FPS, VFR status, and duration:
gemini-video-pipeline probe video.mp4
2. Chunk Video onto Disk
Split a video into scene-aligned, token-budgeted MP4 files:
gemini-video-pipeline chunk video.mp4 \
--output-dir /path/to/my_chunks \
--budget 200000 \
--motion-filter \
--motion-threshold 0.10 \
--max-duration 300.0
CLI options for chunk:
| Option | Default | Description |
|---|---|---|
--output-dir |
./chunks |
Target folder for generated chunk files |
--budget |
200000 |
Maximum token ceiling per video chunk |
--motion-filter / --no-motion-filter |
True |
Filter out static/idle intervals |
--motion-threshold |
0.10 |
Motion saliency cutoff threshold (0.0 to 1.0) |
--max-duration |
300.0 |
Maximum duration in seconds for any single chunk |
3. Analyze Video Chunks with Gemini
Splits video, uploads chunks, and queries Gemini with temporal grounding metadata (requires GEMINI_API_KEY):
export GEMINI_API_KEY="your-api-key"
gemini-video-pipeline analyze video.mp4 --prompt "Identify any security incidents" --model gemini-2.5-flash
Python API Reference
Pipeline Configuration Options (PipelineConfig)
from gemini_video_pipeline import PipelineConfig
config = PipelineConfig(
output_dir="./chunks", # Directory where chunk files are saved
auto_cleanup=True, # Ephemeral deletion of chunk files upon context exit
target_token_budget_per_chunk=200_000, # Gemini token limit per chunk
scene_cut_threshold=0.35, # Chi-Square distance threshold for HARD scene cuts
soft_cut_threshold=0.20, # Chi-Square distance threshold for SOFT scene cuts
motion_filter_enabled=True, # Whether to skip static/idle footage
motion_threshold=0.10, # Mean luminance delta threshold for idle detection
max_chunk_duration_sec=300.0, # Max duration before forcing a chunk boundary
min_chunk_duration_sec=2.0, # Min chunk duration to prevent micro-fragments
sample_rate_fps=1.0, # Base frame sampling rate
upload_to_files_api=True, # Route chunks >= 20 MB through Gemini Files API
)
Chunk Object Attributes (VideoChunk)
Each chunk yielded by pipeline.process(video_path) contains:
chunk.chunk_index: 1-based index of the chunk.chunk.file_path: Local disk path (insideoutput_dir).chunk.start_timestamp/chunk.end_timestamp: Global timestamps formatted asHH:MM:SS.mmm.chunk.start_sec/chunk.end_sec: Start and end times in seconds.chunk.duration_sec: Duration of this chunk.chunk.frame_count: Total video frames in the chunk.chunk.source_fps: Source frame rate.chunk.average_motion_score: Average motion saliency score (0.0to1.0).chunk.estimated_tokens: Estimated Gemini multimodal token consumption.chunk.sampling_plan: Token budget plan withtarget_fpsand optionalscale_resolution.chunk.to_gemini_content(): Converts the chunk directly into atypes.Part(inline bytes or Files API URI).
Autonomous Agent Integration
Google ADK 2.x
from gemini_video_pipeline.tools.adk_tool import VideoChunkerTool
tool = VideoChunkerTool(output_dir="./agent_chunks", auto_cleanup=False)
# Register as function declaration for Gemini function calling:
func_declaration = tool.as_function_declaration()
# Or pass directly as an ADK callable:
agent_tool = tool.as_adk_callable()
manifest = agent_tool(
video_path="security_cam.mp4",
token_budget_per_chunk=150_000,
focus_high_motion_only=True
)
LangChain
from gemini_video_pipeline.tools.langchain_tool import create_langchain_video_chunker_tool
tool = create_langchain_video_chunker_tool(output_dir="./langchain_chunks")
Testing & Verification
-
Run Unit and Integration Tests:
pytest tests/ -v --cov=gemini_video_pipeline --cov-report=term-missing
-
Run Linting and Static Type Checking:
flake8 src tests --max-line-length=120 mypy src
-
Run Self-Contained Examples:
python examples/01_direct_library_usage.py python examples/02_motion_filtered_audit.py python examples/03_adk_agent_workflow.py
-
Test Across All Formats, Codecs & Lengths (including >10min and ~30min):
# Generate standard multi-format test videos (MP4, MOV, MKV, WEBM, AVI, 3s to 3min, VFR, CCTV) python scripts/generate_test_videos.py # Generate long test videos (12-minute multi-scene video and 30-minute narrative video) python scripts/generate_long_videos.py # Process and verify all 12 test videos with output chunks on disk python scripts/test_all_videos.py
-
Test Variable Token Budgets & Chunk Duration Constraints: Run the verification matrix across a spectrum of token budgets (
5kto500ktokens) and chunk duration limits (15sto240s):python scripts/test_variable_budgets_and_chunks.py
Metadata
Release files for gemini-video-pipeline 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| gemini_video_pipeline-0.1.0.tar.gz | 31.1 kB | Without provenance | Details for gemini_video_pipeline-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| gemini_video_pipeline-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for gemini_video_pipeline-0.1.0-py3-none-any.whl |
Total release size: 71.0 kB
Release files / gemini_video_pipeline-0.1.0.tar.gz
| Download URL | gemini_video_pipeline-0.1.0.tar.gz |
|---|---|
| Size | 31.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
57bef8ea3c7fd31ca02bb725736fdb770585d9733f6f82d3f14e9d58d9a3e972
|
|
BLAKE2b-256 checksum How to use checksums |
672639d1826676404adcc6a652b257b85acac6005aad888ea31f6827ae69c16c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.5
|
Release files / gemini_video_pipeline-0.1.0-py3-none-any.whl
| Download URL | gemini_video_pipeline-0.1.0-py3-none-any.whl |
|---|---|
| Size | 39.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
258f72eebbee7d1502d1b0fb856cc4dfdf1cb6cb0637dcb63d4aab522e6d4d9b
|
|
BLAKE2b-256 checksum How to use checksums |
6d084a701caa292e8f5f10fd8e089f9ba988157c616b91aeabd18c131bd4d18a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.5
|
Security for gemini-video-pipeline 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.