pill-id

"What pill is this?" -- Identify pills and medications from images using AI vision models.

$ pill-id identify mystery_pill.jpg

Extracted Features:
  Color: white
  Shape: capsule
  Imprint (front): TYLENOL 500

Top 3 Matches:
  1. Tylenol Extra Strength (Acetaminophen) 500mg
     Manufacturer: Johnson & Johnson
     Confidence: 92%

  2. Tylenol (Acetaminophen) 325mg
     Manufacturer: Johnson & Johnson
     Confidence: 58%

Disclaimer: This tool is for informational purposes only.
Always verify medication identity with a pharmacist.

Why?

Unidentified pills are a real safety problem. Every year, poison control centers receive thousands of calls about unknown medications found in homes, on the ground, or mixed up in pill organizers.

Existing pill identification tools are either closed-source, require paid subscriptions, or rely on outdated databases. pill-id is the first open-source pill identification library that combines state-of-the-art AI vision models with an embedded medication database.

How It Works

pill-id uses a two-phase approach:

1. Vision AI Extraction: Send a pill image to GPT-4V or Claude Vision to extract visual features (color, shape, imprint text, scoring marks, coating type)
2. Database Matching: Match extracted features against an embedded database of 200+ common medications, scored by confidence

This approach is more practical than training a custom image classifier because:

• No training data needed
• Works immediately with any new pill
• Leverages state-of-the-art vision models
• Easy to extend the database

Quick Start

Installation

pip install pill-id

For Anthropic Claude support:

pip install pill-id[anthropic]

Python API

from pill_id import PillIdentifier

identifier = PillIdentifier(
    provider="openai",      # or "anthropic"
    api_key="sk-...",       # or set OPENAI_API_KEY env var
)

# Identify from a file
result = identifier.identify("pill_photo.jpg")
print(result.matches[0].name)          # "Acetaminophen 500mg"
print(result.matches[0].confidence)    # 0.92
print(result.extracted_features.color) # "white"

# Identify from URL
result = identifier.identify("https://example.com/pill.jpg")

# Identify from bytes
with open("pill.jpg", "rb") as f:
    result = identifier.identify(f.read())

# Batch identification
results = identifier.identify_batch(["pill1.jpg", "pill2.jpg"])

CLI

# Identify a pill
pill-id identify photo.jpg
pill-id identify photo.jpg --provider anthropic --top 5
pill-id identify photo.jpg --json-output

# Search the database
pill-id search --imprint "M 357" --color white
pill-id search --imprint "TYLENOL" --json-output

# Database stats
pill-id database

Supported Providers

Provider	Model	Env Variable
OpenAI	GPT-4o (default)	OPENAI_API_KEY
Anthropic	Claude Sonnet (default)	ANTHROPIC_API_KEY

Database

The embedded database includes 200+ medications:

• 100+ top prescribed US medications (by visual appearance)
• 30+ common OTC medications (Tylenol, Advil, Aleve, etc.)
• 50+ popular supplements (Vitamin D, Fish Oil, Magnesium, etc.)
• 50 manufacturer imprint patterns (M = Mylan, IP = Amneal, etc.)

Custom Database

You can supply your own pill database:

from pill_id import PillIdentifier
from pill_id.models import PillDatabaseEntry

custom_db = [
    PillDatabaseEntry(
        name="My Custom Med",
        generic_name="Customamine",
        strength="50mg",
        manufacturer="Custom Pharma",
        imprint="CP 50",
        color="blue",
        shape="round",
    ),
]

identifier = PillIdentifier(
    provider="openai",
    api_key="sk-...",
    custom_database=custom_db,
)

Confidence Scoring

Matches are scored based on multiple features:

Feature	Weight
Exact imprint match	+0.50
Partial imprint match	+0.25
Color match	+0.15
Shape match	+0.15
Manufacturer code match	+0.10
Scoring/coating match	+0.10

Safety

IMPORTANT: This tool is for informational purposes only. Always verify medication identity with a pharmacist or healthcare provider. Never take medication based solely on AI identification.

pill-id includes built-in safety features:

• Every result includes a disclaimer
• Low-confidence matches (< 70%) trigger explicit warnings
• Multiple ambiguous matches are flagged
• Controlled substances are flagged with additional warnings
• The library never suggests dosage or usage information

Accuracy Notes

• Best results come from clear, well-lit photos of a single pill against a contrasting background
• Imprint text is the strongest identifier -- if the model can read the imprint, accuracy is high
• Color and shape alone are unreliable -- many different medications share the same appearance
• Results should always be verified by a pharmacist
• The database covers common US medications; international medications may not be included

Development

# Clone and install
git clone https://github.com/pillright/pill-id.git
cd pill-id
pip install -e ".[dev,anthropic]"

# Run tests (no API keys needed)
pytest

# Lint
ruff check src/ tests/

Related Projects

• pillright/pharma-ai-agent -- AI-powered medication safety checker
• pillright/rxnorm-embeddings -- RxNorm medication name embeddings
• pillright/drug-interaction-api -- Drug interaction checking API

Contributing

Contributions are welcome! Please see our contributing guidelines:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/my-feature)
3. Add tests for any new functionality
4. Ensure all tests pass (pytest)
5. Submit a pull request

License

MIT License. See LICENSE for details.