annotools

Let agents see and annotate multimodal data within a token budget.

Documentation · 中文

Feeding full-resolution media to a multimodal model is expensive and imprecise. annotools gives an agent purpose-built views instead — downscaled previews, crop-zoom, grid guide lines, and overlays of boxes, keypoints, polygons and segmentation masks so it can check its own annotations before committing them. Every coordinate stays in one convention: normalized 0.0–1.0 relative to the uncropped source.

Two ways to use it

Quick start

uv add "annotools[media] @ git+https://github.com/hoshiori-dev/annotools"

Pre-1.0: install from git. The first full release publishes to PyPI; every release also goes to TestPyPI, which rehearses the publishing path and is not an install channel. The media extra adds PyAV, which the video and audio tools need; drop it for images only.

As an MCP server

Register the annotools command over stdio. For Claude Code, .mcp.json:

{
  "mcpServers": {
    "annotools": {
      "type": "stdio",
      "command": "uv",
      "args": ["run", "annotools"],
      "env": { "ANNOTOOLS_MAX_WIDTH": "768", "ANNOTOOLS_MAX_HEIGHT": "768" }
    }
  }
}

Set the preview size for the model behind the agent: 384 px (the default) keeps a Gemini image at one 258-token unit, while Claude and GPT bill by area and read 768 px comfortably. Codex and OpenCode shapes, every setting, and the HTTP transport are in Register the server.

As a library

from annotools import BBoxObject, draw_bboxes, encode, load_image, normalize_coordinates, preview

result = preview(load_image("photo.jpg"), max_width=768, max_height=768)
boxes = normalize_coordinates(
    [[240, 130, 470, 505]],
    result.metadata["output_width"],
    result.metadata["output_height"],
)
overlay = draw_bboxes(result, [BBoxObject(bbox=boxes[0], label="cat")])
jpeg = encode(overlay.image, "jpeg")

Tools

Parameters, return shapes and specifications: tool reference.

Documentation

• Get started — install, extras, container, MCP registration.
• Usage — settings, coordinate conventions, the shape of a library call.
• API reference — every public function, generated from its docstring.
• Architecture — layers and recorded decisions.
• skills/ — the annotation methodology, installable into your agent: npx skills add hoshiori-dev/annotools.
• examples/ — four complete pipelines: captioning and detection, on the Claude Agent SDK and the Codex SDK.

Development

uv sync --all-extras
just check
just docker-build && just test-container

just check runs lint, format, types, taxonomy, README sync, the public-API docstring check, the generated-reference drift check, a strict docs build, and the tests behind a 95% coverage gate. See CONTRIBUTING.md; agents start from AGENTS.md.

License

Apache-2.0