describe-it

Alt-text descriptions for PIL images, from a vision model running on your own machine.

What it is

describe-it takes a PIL.Image.Image and returns a short, alt-text-quality description of it as a plain string, by asking a vision model served by Ollama — on your own machine by default, and no third party at any point. That matters for two reasons: generating alt text at volume is cheap locally and expensive through a commercial API, and the images may be NSFW, which hosted services refuse or penalise — a local model has no policy layer beyond whatever is in its weights.

The contract is deliberately small: image in, description out, every failure is an exception. No result objects, no status codes, no None returns.

See ARCHITECTURE.md for how it is put together.

Requirements

• Python 3.12 or newer.

• A running Ollama with a vision-capable model pulled:

  $ ollama pull qwen3.5:4b
  

Pillow is the only runtime dependency; the Ollama transport is urllib from the standard library.

Install

The distribution on PyPI is describe-image (the name describe-it was already taken there). The import package is still describe_it and the command is still describe-it:

$ uv add describe-image
$ pip install describe-image

To work on it, clone and sync. uv sync creates the project's own virtual environment in .venv/ and installs describe-it into it, so run things through uv run rather than expecting the package on your system Python:

$ git clone git@github.com:Xof/describe-it.git
$ cd describe-it
$ uv sync
$ uv run describe-it --help

To track the repository instead of a release, install straight from it:

$ uv add git+https://github.com/Xof/describe-it
$ pip install git+https://github.com/Xof/describe-it

Usage

The zero-configuration call:

from PIL import Image
from describe_it import describe

alt = describe(Image.open("cat.jpg"))
# -> "A grey tabby cat asleep on a red armchair beside a sunlit window."

Any mode Pillow can open works — RGBA, palette GIFs, CMYK TIFFs, 16-bit scientific images, sideways phone photographs. You never have to convert first, and your image object is not modified.

For a batch, configure once and keep the model resident:

from describe_it import Describer

d = Describer(model="gemma4:e4b", max_words=20, keep_alive="1h")
d.ensure_model()  # downloads the model if it is missing (explicit, opt-in)
for img in images:
    alt = d.describe(img, context="gallery thumbnail")

Every failure is an exception, and they all share a base class:

from describe_it import DescribeItError, DescriptionRefusedError

try:
    alt = describe(image)
except DescriptionRefusedError as exc:
    # The model declined. Its actual words are on the exception.
    print(f"refused: {exc.response}")
except DescribeItError as exc:
    print(f"no description: {exc}")

Options

Both describe() and Describer() take these; context and prompt are per-call, so they are arguments to Describer.describe() rather than to its constructor.

Exceptions

Everything the library raises inherits from DescribeItError, so one except clause catches all of it. A wrong-type argument is the deliberate exception: a non-image raises plain TypeError, and out-of-range options raise ValueError.

DescribeItError
├── ImageError                  the image cannot be prepared (zero size, unloadable)
├── OllamaError                 anything involving the server
│   ├── OllamaConnectionError   cannot connect (refused, DNS, unreachable)
│   ├── OllamaTimeoutError      connecting or reading exceeded the timeout (a sibling, not a subclass)
│   ├── ModelNotFoundError      the server does not have the model; names the pull command
│   └── OllamaResponseError     any other non-2xx, or an unusable body; has .status_code and .body
└── DescriptionError            the server answered but produced no usable description
    └── DescriptionRefusedError the model refused; .response holds its actual words

Environment variables

Both are read when a describer is constructed — which is per call for describe(), and once for a long-lived Describer — never at import time. A blank value counts as unset.

Command line

describe-it [--model TAG] [--host URL] [--timeout SECONDS] [--context TEXT]
            [--language NAME] [--max-words N] [--max-image-size PX]
            [--version] FILE [FILE ...]

One file prints the description alone; several print <path>\t<description> per line. A file that cannot be read or described is reported on stderr and the run continues, with exit status 1 at the end if anything failed.

$ describe-it photo.jpg
A blue and yellow flag against a bright sky.

$ describe-it --max-words 15 photo.jpg diagram.png
photo.jpg	A blue and yellow flag against a bright sky.
diagram.png	A flow chart with three boxes joined by arrows.

Choosing a model

Any vision-capable Ollama model works; the tag is passed straight through. The default is qwen3.5:4b (~3 GB), chosen because Qwen models are generally less refusal-prone than the Gemma line — which matters when the images are the sort a hosted service would reject. No measurement backs that up; it is the judgement recorded when the default was chosen.

• gemma4:e4b — reportedly better captions, and more likely to refuse. Also unmeasured.
• llava:7b — older and widely available. In this project's live runs it answered in English whatever language asked for, and overran max_words in two runs out of seven.

If the default is too large for your machine, Ollama's library lists smaller vision tags; none of them have been tried here.

A model that declines to describe an image produces DescriptionRefusedError rather than alt text, so a refusal-prone model turns into exceptions rather than into garbage on your web pages.

Development

$ uv sync --locked --group dev

The four commands CI gates on, which should all be run before pushing:

$ uv run ruff check .
$ uv run ruff format --check .
$ uv run mypy src tests
$ uv run pytest --cov --cov-report=term-missing -m "not integration"

The unit suite is hermetic — no Ollama, no model — and the coverage gate is 100%.

The tests under tests/integration/ talk to a real server and a real model. They are opt-in, skipped unless both the environment variable is set and the host answers:

$ DESCRIBE_IT_INTEGRATION=1 DESCRIBE_IT_MODEL=qwen3.5:4b \
      uv run pytest -m integration -v

Their assertions are structural (length, language, exception type), because a small vision model's output is not deterministic enough to gate a merge on.