image-curator 0.1.0

A production-ready Python tool for curating and downloading images from multiple sources

Image Curator

A production-ready, modular Python application for curating and downloading images from multiple sources based on configurable category searches. Currently supports Flickr with an extensible plugin architecture designed for easy addition of new image sources.

💡 Note on Flickr API: As of 2024, Flickr API requires a paid Pro subscription. We're planning to add support for free alternatives like Unsplash and Pexels soon. Contributions welcome!

Features

• Multi-source support: Currently supports Flickr, easily extensible to other platforms
• Category-based downloads: Define categories in config and download images for each
• Smart filtering: Control license type, safe search, and content type
• Progress tracking: Real-time progress bars with tqdm
• Robust error handling: Automatic retries with exponential backoff
• Clean file organization: Images saved in per-category subdirectories with sanitized names
• Flexible configuration: Override settings via CLI or config file
• Plugin architecture: Add new image sources without modifying core code
• Production-quality: Type hints, comprehensive logging, and full test coverage

Project Structure

image-curator/
├── config.py                 # User configuration
├── downloader.py            # Main dispatcher logic
├── image_sources/           # Provider plugins
│   ├── __init__.py         # Provider registry
│   └── flickr.py           # Flickr implementation
├── utils.py                # Helper functions
├── main.py                 # CLI entry point
├── requirements.txt        # Dependencies
├── README.md              # This file
├── LICENSE                # MIT License
└── tests/                 # Test suite
    ├── test_config.py
    ├── test_flickr_downloader.py
    └── __init__.py

Requirements

• Python 3.8 or higher
• Flickr API key (Note: As of 2024, Flickr API requires a paid subscription - see setup details below)

Installation

Option 1: Install from PyPI (Recommended)

pip install image-curator

Option 2: Install from Source

1. Clone the repository

git clone https://github.com/ghostcipher1/image-curator.git
cd image-curator

2. Create a virtual environment

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install dependencies

pip install -r requirements.txt
# Or install in development mode
pip install -e .

Configuration

Get a Flickr API Key

Important: As of 2024, Flickr API access requires a paid Flickr Pro subscription ($8.25/month or $71.99/year).

1. Subscribe to Flickr Pro: Visit Flickr Pro and subscribe
2. Request API Key: Go to Flickr App Garden
3. Click "Request an API Key"
4. Choose "Apply for a Non-Commercial Key" (for personal use) or "Commercial Key" (for business use)
5. Fill out the form:
   • App name: "Image Curator"
   • App description: "Personal image collection and curation tool"
6. Copy your API Key (you don't need the secret for this application)

Configure Your API Key

Option A: Edit config.py (recommended for personal use)

API_KEYS = {
    "flickr": "your_actual_flickr_api_key_here"
}

Option B: Use environment variable (recommended for production)

Create a .env file in the project root:

FLICKR_API_KEY=your_actual_flickr_api_key_here

Or export it in your shell:

export FLICKR_API_KEY=your_actual_flickr_api_key_here

Configuration

Edit config.py to customize your downloads:

# Categories to download images for
CATEGORIES = ["sunsets", "mountains", "wildlife"]

# Number of images to download per category
NUM_IMAGES = 10

# Output directory for downloaded images
OUTPUT_DIR = "downloads"

# Which provider to use
SOURCE = "flickr"

# Flickr-specific options
FLICKR_CONFIG = {
    "license": "any",      # License filter (see below)
    "safe_search": 1,      # 1=safe, 2=moderate, 3=restricted
    "content_type": 1      # 1=photos, 2=screenshots, 3=other
}

Flickr License Options

The license parameter filters results by license type:

• "any" - All licenses (default)
• "0" - All Rights Reserved
• "4" - Attribution License (CC BY)
• "5" - Attribution-ShareAlike License (CC BY-SA)
• "6" - Attribution-NoDerivs License (CC BY-ND)
• "7" - No known copyright restrictions
• "8" - United States Government Work
• "9" - Public Domain Dedication (CC0)
• "10" - Public Domain Mark

For Creative Commons images suitable for reuse, consider licenses 4, 5, 6, 9, or 10.

Usage

Basic usage (uses config.py settings)

python main.py

Download specific number of images

python main.py --limit 20

Override categories

python main.py --categories "cats,dogs,birds"

Specify output directory

python main.py --out my_images

Filter by Creative Commons license

python main.py --license 4

Combined options

python main.py --categories "nature,landscapes" --limit 15 --safe_search 1 --license any

Enable verbose logging

python main.py --verbose

Full CLI options

usage: main.py [-h] [--source SOURCE] [--limit LIMIT] [--out OUT]
               [--categories CATEGORIES] [--safe_search {1,2,3}]
               [--license LICENSE] [--content_type {1,2,3}] [--verbose]

Options:
  --source SOURCE            Image source to use (default: flickr)
  --limit LIMIT             Number of images per category (default: 10)
  --out OUT                 Output directory (default: downloads)
  --categories CATEGORIES   Comma-separated categories
  --safe_search {1,2,3}     Flickr safe search level
  --license LICENSE         Flickr license filter (e.g., '4', '5', 'any')
  --content_type {1,2,3}    Flickr content type: 1=photos, 2=screenshots, 3=other
  --verbose, -v             Enable verbose logging (DEBUG level)

Output Structure

Images are organized in subdirectories by category:

downloads/
├── sunsets/
│   ├── sunsets_001__12345678.jpg
│   ├── sunsets_002__23456789.jpg
│   └── ...
├── mountains/
│   ├── mountains_001__34567890.jpg
│   └── ...
└── wildlife/
    └── ...

Filename format: {category}_{index}__{photo_id}.{ext}

Testing

Run the test suite:

pytest

Run with coverage:

pytest --cov=. --cov-report=html

Run specific test file:

pytest tests/test_flickr_downloader.py -v

Tests use mocked API responses and do not make real network calls.

Important Legal & Ethical Notes

Respect Flickr's Terms of Service

• This tool uses the official Flickr API
• Follow Flickr's API Terms of Use
• Respect rate limits (3600 queries per hour per API key)
• Do not use this tool for scraping or bulk downloading beyond reasonable personal use

Check Image Licenses

• Always check the license before using downloaded images
• Creative Commons licenses have specific requirements (attribution, etc.)
• "All Rights Reserved" means you cannot use the image without permission
• This tool downloads images but does not grant you usage rights
• See Creative Commons License Types for details

Respect Photographers

• Give proper attribution when required by license
• Consider donating or purchasing prints from photographers whose work you use
• Do not claim others' work as your own

Extensibility

The architecture is designed to support multiple image sources. To add a new provider:

1. Create image_sources/new_provider.py
2. Implement a class with __init__(api_key, config) and download(query, num_images, output_dir)
3. Register in image_sources/__init__.py PROVIDERS dict
4. Add provider-specific config to config.py

Example skeleton:

# image_sources/new_provider.py
class NewProviderDownloader:
    def __init__(self, api_key: str, config: dict = None):
        self.api_key = api_key
        self.config = config or {}

    def download(self, query: str, num_images: int, output_dir: str) -> None:
        # Implement search and download logic
        pass

Troubleshooting

"No API key found for flickr"

• Ensure you've set your API key in config.py or as environment variable FLICKR_API_KEY

"Flickr API error: Invalid API Key"

• Check that your API key is correct and active
• Verify it's not expired on Flickr App Garden

"No photos found for query"

• Try different search terms
• Adjust license/safe_search filters (some combinations return fewer results)
• Check Flickr.com directly to verify photos exist for your query

Rate limit errors

• Flickr allows 3600 API calls per hour
• Reduce NUM_IMAGES or wait before retrying

Connection/timeout errors

• Check your internet connection
• Increase timeout in utils.py http_get_with_retry function
• Retry the download (tool automatically retries with backoff)

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Write tests for new functionality
4. Ensure all tests pass (pytest)
5. Follow PEP 8 style guidelines
6. Commit with clear messages
7. Push to your branch
8. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Flickr API for providing free access to their photo database
• All photographers who share their work on Flickr

Disclaimer

This tool is provided as-is for educational and personal use. Users are responsible for:

• Complying with Flickr's Terms of Service
• Respecting image licenses and copyright
• Obtaining necessary permissions for commercial use
• Proper attribution where required

The authors assume no liability for misuse of downloaded images.