AutoVQA

A Python library for automated visual question answering data generation.

---

Installation

pip install autovqa

For development installation:

git clone https://github.com/Ddyln/AutoVQA.git
cd AutoVQA
pip install -e ".[dev]"

Quick Start

import json
from autovqa.evaluate import run_evaluate
from autovqa import eda_pipeline, filter_pipeline, balancer_pipeline

# Step 1: Evaluate your augmented VQA data (supports Google Gemini, OpenAI, QwenVL)
run_evaluate(
    text_path="output/augmented_vqa.json",
    image_path="data/processed_images",
    output_dir="output",
    service="google",  # Options: "google", "openai", "qwenvl"
    model="gemini-2.5-flash",
    limit_samples=-1,
    max_retries=3,
)

# Step 2: Load the evaluated data
with open("output/raw_answers.json", "r", encoding="utf-8") as f:
    data = json.load(f)

# Step 3: Run EDA pipeline
df = eda_pipeline(data, output_dir="./reports", generate_report=True)

# Step 4: Filter low-quality records
df = filter_pipeline(df, threshold=0.5)

# Step 5: Balance the dataset
df = balancer_pipeline(df, output_path="./balanced.csv")

Pipeline Overview

Collect ──► Augment ──► Evaluate ──► EDA ──► Filter ──► Balance

Step	Module	Output
1	collect	Raw images + text JSON
2	augment	augmented_vqa.json — new QA pairs
3	evaluate	evaluated_vqa.json — QA + scores
4	eda	Cleaned DataFrame + Excel reports
5	filter	Quality-filtered DataFrame
6	balance	balanced_data.csv

Features

Collect

Download VQA datasets and images from various sources.

from autovqa.collect import download_default_data

download_default_data(output="./data")

Preprocess

Image preprocessing pipeline for VQA tasks (resize, denoise, color correction, sharpening).

from autovqa.preprocess.main import preprocess_image, run_pipeline

# Single image
processed = preprocess_image("image.jpg", target_size=(480, 640))

# Batch processing
run_pipeline(input_folder="./raw_images", output_folder="./processed_images")

Augment

Generate VQA question-answer pairs using LLMs (e.g., Gemini).

from autovqa.augment.client import AugmentClient

client = AugmentClient(service_name="google")
results = client.run_pipeline(
    image_folder_dir="./images",
    output_json_path="./augmented_vqa.json"
)

Evaluate

Score each QA pair and its image across multiple dimensions using multimodal LLM service providers (Google Gemini, OpenAI ChatGPT, or local Qwen2.5-VL). You can select your provider using the service parameter.

from autovqa.evaluate import run_evaluate

# Evaluate records using Google Gemini, OpenAI, or QwenVL
run_evaluate(
    text_path="./augmented_vqa.json",
    image_path="./images",
    output_dir="./output",
    service="google",  # Options: "google", "openai", "qwenvl"
    model="gemini-2.5-flash",
    limit_samples=-1,
    max_retries=3
)

Scores produced per record:

• eip_ — Image quality (clarity, occlusion, object density, scene clutter, etc.)
• idp_ — Image diversity (scene type, main object, cultural context, demographics)
• etp_ — Text quality (grammar, unambiguity, QA structure, syntactic complexity)
• vqac_ — VQA correlation (question↔image, answer↔image, reasoning depth)

EDA (Exploratory Data Analysis)

Analyze evaluated VQA data with cleaning, feature extraction, and report generation.

from autovqa import eda_pipeline

df = eda_pipeline(
    data=data,
    output_dir="./reports",
    generate_report=True,
    aggregation_type="median"
)

Filter

Filter data based on quality labels and thresholds.

from autovqa import filter_pipeline

df_filtered = filter_pipeline(df, threshold=0.5, show_stats=True)

Balance

Balance class distributions in your dataset.

from autovqa import balancer_pipeline

df_balanced = balancer_pipeline(df_filtered, output_path="./balanced.csv")

Data Format

AutoVQA expects JSON data in the following format for the EDA/Filter/Balance pipeline (output of the Evaluate step):

[
    {
        "question": "What is in the image?",
        "answers": ["Answer 1", "Answer 2", "Answer 3", "Answer 4", "Answer 5"],
        "image_link": "http://images.cocodataset.org/...",
        "image_name": "000000001.jpg",
        "index": 1,
        "image_quality_evaluation": { "...": "..." },
        "image_diversity_evaluation": { "...": "..." },
        "text_quality_evaluation": { "...": "..." },
        "correlation_evaluation": { "...": "..." }
    }
]

Documentation

See the Getting Started Notebook for detailed usage examples, or the Example Notebook for a minimal end-to-end run.

Development

For more details on setting up a development environment, please refer to the Development Guide.

Contributing

Contributions are welcome! Please read our Contributing Guide for details.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Citation

If you use AutoVQA in your research, please cite:

@misc{tuong2026autoviVQA,
  title  = {AutoViVQA: A Large-Scale Automatically Constructed Dataset for Vietnamese Visual Question Answering},
  author = {Nguyen Anh Tuong, Phan Ba Duc, Nguyen Trung Quoc, Tran Dac Thinh, Dang Duy Lan, Nguyen Quoc Thinh, Tung Le},
  year   = {2026},
  eprint = {2603.09689},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url    = {https://arxiv.org/abs/2603.09689}
}