Amon Hen

"From the Seat of Seeing, no moment remains hidden."

A fast, lightweight CLI and Python library for natural language video moment retrieval on local CPU. Runs entirely on CPU without discrete GPUs, background daemons, or cloud dependencies.

Key Highlights

• 100% CPU Execution: Powered by Apple's MobileCLIP2 architecture, Whisper-Tiny ONNX, and ONNX Runtime.
• Hybrid Visual & Speech Retrieval: Simultaneously retrieves moments based on visual semantics and spoken dialogue.
• Low Footprint: Hybrid FP32-vision + INT8-quantized text pipeline (approx. 105 MB RAM).
• Zero Infrastructure: Embedded vector database and full-text index via SQLite, sqlite-vec, and FTS5 (single .db file).
• High Throughput: 3-gate adaptive motion sampler achieving 4.8x to 18.5x realtime indexing speed.
• Temporal Grouping: Merges consecutive matching frames into start-end timestamp intervals.

Installation

# Recommended via uv
uv tool install amon-hen

# Or via pipx
pipx install amon-hen

# Or standard pip
pip install amon-hen

Quickstart

1. Interactive Terminal UI

Run amon-hen without arguments to launch the interactive prompt:

amon-hen

Supports instant search queries, history navigation (↑/↓), /open <n> to play matches in your default player, and /cut <n> [output] to export clips.

2. Index Videos

# Index a video or entire folder with adaptive motion sampling
amon-hen index /path/to/videos/ --sampler adaptive

3. Search Moments

# Search visual actions or spoken dialogue (Battle of Amon Hen)
amon-hen search "swords fight warriors in forest"

# Search real-world surveillance events (CCTV footage)
amon-hen search "a person holding an umbrella"

4. Extract Video Clip

# Extract matching segment into a standalone video clip
amon-hen cut battle-of-amon-hen.webm --start 00:03:56 --end 00:04:52 -o battle_climax.mp4

Python API

from amonhen.encode import TextEncoder
from amonhen.model_registry import get_model
from amonhen.pipeline import search
from amonhen.store import Store

# 1. Connect to index
store = Store("index.db", embed_dim=512)

# 2. Load CPU-optimized text encoder
spec = get_model("mobileclip2-s0")
text_encoder = TextEncoder(spec)

# 3. Retrieve matching video segments
results = search("red sports car turning", store=store, text_encoder=text_encoder)
for seg in results:
    print(
        f"{seg.video_path}: {seg.start_ms / 1000.0:.1f}s - {seg.end_ms / 1000.0:.1f}s (score: {seg.score:.3f})"
    )

store.close()

Capabilities & Known Limitations

• Visual Semantic Search (MobileCLIP2): Strong zero-shot recognition for objects, actions, settings, spatial arrangements, and characters. Adding descriptive context (e.g. "Aragorn with sword" vs "Aragorn") maximizes cosine similarity and avoids domain noise.
• Speech-to-Text (Whisper-Tiny ONNX): Indexes clean spoken dialogue across speeches, podcasts, lectures, and vlogs. It is not an acoustic sound-effects classifier. Low-confidence dialogue on heavy orchestral soundtracks is automatically filtered to prevent hallucinations.
• Lossless Stream Cut: Sub-second extraction aligns to nearest container keyframes; use --reencode for exact frame-level cuts.

Resources & Links

• GitHub Repository: https://github.com/flxhrdyn/amon-hen

• CLI User Manual: https://github.com/flxhrdyn/amon-hen/blob/main/docs/CLI_GUIDE.md

• Python API Documentation: https://github.com/flxhrdyn/amon-hen/blob/main/docs/PYTHON_API.md

• Project Roadmap: https://github.com/flxhrdyn/amon-hen/blob/main/docs/ROADMAP.md

• Hugging Face Model Hub: https://huggingface.co/flxhrdyn/mobileclip2-s0-onnx

• License: MIT License