bulk-webp-url-replacer

Bulk convert images to WebP and automatically update URLs in markdown files with a custom CDN prefix.

Features

• 🔍 Extract image URLs from markdown files (frontmatter, galleries, inline images)
• 📥 Download images from remote URLs (parallel downloads)
• 🖼️ Convert to optimized WebP format
• 🔄 Replace original URLs with new CDN-prefixed paths
• ⏭️ Skip already-processed images and excluded extensions
• 👀 Dry-run mode to preview changes

Installation

pip install bulk-webp-url-replacer

Or install from source:

git clone https://github.com/HoangYell/bulk-webp-url-replacer.git
cd bulk-webp-url-replacer
pip install -e .

Usage

CLI

# Dry run - preview what would be processed
bulk-webp-url-replacer \
  --scan-dir ./content \
  --output-dir ./webp_images \
  --dry-run

# Full run with custom URL prefix
bulk-webp-url-replacer \
  --scan-dir ./content \
  --output-dir ./webp_images \
  --new-url-prefix "https://cdn.example.com/images"

# Faster with more threads
bulk-webp-url-replacer \
  --scan-dir ./content \
  --output-dir ./webp_images \
  --new-url-prefix "https://cdn.example.com/images" \
  --threads 8

As Python Module

python -m bulk_webp_url_replacer \
  --scan-dir ./content \
  --output-dir ./webp_images \
  --new-url-prefix "https://cdn.example.com/images"

Programmatic Usage

from bulk_webp_url_replacer import ImageETL, ImageURLExtractor

# Full ETL pipeline
etl = ImageETL(
    content_dir="./content",
    webp_dir="./webp_images",
    webp_base_url="https://cdn.example.com/images",
    quality=80,
    max_width=1200,
    exclude_extensions=["gif", "svg", "webp", "ico"],
    threads=4
)

# Dry run to preview changes
result = etl.run(dry_run=True)
print(f"Found {result.total_urls} URLs, {result.skipped} already processed")

# Full run
result = etl.run(dry_run=False)
print(f"Converted {result.converted} images, {result.failed} failed")

# Or just extract URLs without processing
extractor = ImageURLExtractor()
urls = extractor.extract_from_directory("./content")
for file_path, line_num, url in urls:
    print(f"{file_path}:{line_num} -> {url}")

Options

Supported Patterns

The tool detects image URLs in:

# YAML frontmatter
---
image: "https://example.com/image.jpg"
---

# TOML frontmatter
+++
image = "https://example.com/image.jpg"
+++

# Gallery shortcodes
{{< gallery >}}
- https://example.com/photo1.jpg
- https://example.com/photo2.png
{{< /gallery >}}

# HTML img tags in shortcodes
{{< embed >}}
<img src="https://example.com/image.jpg" width="250" height="250"/>
{{< /embed >}}

# Standard markdown
![Alt text](https://example.com/image.jpg)

Output

After running, you'll have:

1. WebP images in your --output-dir
2. mapping.json tracking original → WebP conversions
3. Updated files with new URLs

License

MIT