otat 0.1.0

Vision-Language Model Interpretability Analysis - One Token at a Time

Interpretability Core

Vision-Language Model Interpretability Analysis toolkit for analyzing attention patterns in models like LLaVA and Qwen-VL.

Installation

From PyPI (when published)

pip install otat

From GitHub

pip install [e-mail]:varungupta31/otat_api.git

Local Development

git clone https://github.com/varungupta31/otat_api.git
cd otat_api
pip install -e .

Quick Start

from interpretability.api.wrapper import InterpretabilityAnalyzer

# Initialize analyzer
analyzer = InterpretabilityAnalyzer(
    model_type="llava_onevision",
    model_id="llava-hf/llava-onevision-qwen2-0.5b-ov-hf"
)

# Run analysis
result = analyzer.analyze(
    image_path="path/to/image.jpg",
    task_text="What is in this image?",
    instruction="Answer briefly.",
    blocking_mode="none",
    num_tokens=20
)

print(result['output_tokens'])
print(result['series'])  # Attention patterns

Features

• 🔍 Attention pattern analysis for VLMs
• 🎯 Support for LLaVA, Qwen-VL, and Qwen2-LLM
• 🚫 Attention blocking experiments
• 📊 Token-level attention aggregation

License

MIT

---

## **3. Verify Your Project Structure**

Make sure it looks like this:

otat_api/ (or interpretability-core/) ├── pyproject.toml # ✅ Just created ├── README.md # ✅ Just created ├── LICENSE # Optional but recommended ├── .gitignore # Recommended ├── src/ │ └── interpretability/ │ ├── init.py # ⚠️ Make sure this exists! │ ├── api/ │ │ ├── init.py │ │ └── wrapper.py │ ├── models/ │ │ ├── init.py │ │ ├── base_handler.py │ │ ├── llava_onevision.py │ │ ├── qwen2_llm.py │ │ └── qwen_25_vl.py │ ├── tasks/ │ │ ├── init.py │ │ ├── base_task.py │ │ ├── web_api_task.py │ │ └── ... │ ├── analysis/ │ │ ├── init.py │ │ ├── extractor_aggregated.py │ │ └── ... │ ├── utils/ │ │ ├── init.py │ │ └── ... │ └── prompts/ │ └── ... └── scripts/ └── run_analysis.py