Merleau

"The world is not what I think, but what I live through." — Maurice Merleau-Ponty

A CLI tool for video understanding using Google's Gemini API. Named after Maurice Merleau-Ponty, the phenomenologist philosopher whose work on perception inspires how this tool helps you perceive your videos.

https://github.com/user-attachments/assets/e2c5b476-ddab-49ab-a35c-9ae5e880c25c

Why Merleau?

Google Gemini is the only major AI provider with native video understanding—Claude doesn't support video, and GPT-4o requires frame extraction workarounds. Merleau is the first CLI that actually understands video rather than analyzing frames.

Features

• Native Gemini video processing - Upload and analyze videos directly
• YouTube URL support - Analyze videos directly from YouTube (free preview)
• Customizable prompts - Ask any question about your video
• Cost estimation - Token usage tracking and cost breakdown
• Multiple models - Support for different Gemini models
• Web UI - Streamlit app for browser-based analysis

Use cases

Clone apps

Take a screencast of your app and ask:

• "What are the main features of this app?"
• "What are the main UI elements?"
• "What are the main user flows?"

Extract code from a coding screencast

ponty https://www.youtube.com/watch?v=Be0ceKN81S8 -p "Extract the text in the first claude code session" -e md

Installation

Using uv (recommended):

uv sync

Or install from PyPI:

pip install merleau

Configuration

1. Get a Gemini API key from Google AI Studio
2. Set the API key as an environment variable or create a .env file:

   GEMINI_API_KEY=your_api_key_here
   

Usage

# Basic video analysis
ponty video.mp4

# Analyze a YouTube video directly
ponty https://youtu.be/VIDEO_ID
ponty https://www.youtube.com/watch?v=VIDEO_ID

# Custom prompt
ponty video.mp4 -p "Summarize the key points in this video"

# Use a different model
ponty video.mp4 -m gemini-2.0-flash

# Export analysis to markdown
ponty video.mp4 -e md

# Hide cost information
ponty video.mp4 --no-cost

Web UI

Try it online: https://merleau.streamlit.app/

The web app supports both file uploads and YouTube URLs (paste a URL in the YouTube tab to preview and analyze directly).

Or run locally:

pip install merleau[web]
streamlit run streamlit_app.py

Options

Reducing Costs with Compression

Compressing videos before analysis can reduce API costs by ~10-15% without degrading analysis quality. Gemini's token count is affected by video resolution and bitrate.

Quick Compression with ffmpeg

# Basic compression (recommended)
ffmpeg -i input.mp4 -vcodec libx264 -crf 28 -preset medium -vf "scale=1280:-2" output.mp4

# Aggressive compression (smaller file, lower quality)
ffmpeg -i input.mp4 -vcodec libx264 -crf 32 -preset medium -vf "scale=640:-2" output.mp4

# Keep audio (for speech analysis)
ffmpeg -i input.mp4 -vcodec libx264 -crf 28 -preset medium -vf "scale=1280:-2" -acodec aac -b:a 128k output.mp4

Compression Options Explained

Cost Comparison Example

Output

The CLI provides:

• Video content analysis from Gemini
• Token usage breakdown (prompt, response, total)
• Estimated cost based on Gemini pricing

Pricing Reference

Gemini 2.5 Flash (as of 2025):

• Input: $0.15 per 1M tokens (text/image), $0.075 per 1M tokens (video)
• Output: $0.60 per 1M tokens, $3.50 for thinking tokens

A 1-hour video costs approximately $0.11-0.32 to analyze.