imagebot 1.0.2

A web bot to scrape images from websites.

A web bot to scrape images from websites.

Features

• Supported platform: Linux (+Gnome) / Python 2.x.

• Uses scrapy web crawling framework.

• Maintains a database of all downloaded images to avoid duplicate downloads.

• Optionally, it can scrape only under a particular url, e.g. scraping “http://website.com/albums/new” with this option will only download from new album.

• You can specify minimum image size to be downloaded.

• Scrapes through javascript popup links.

• Live monitor window for displaying images as they are scraped.

Usage

1. Scrape images from http://website.com:

   imagebot http://website.com

2. Scrape images from http://website.com while allowing images from a cdn such as amazonaws.com (add multiple domains with comma separated list):

   imagebot http://website.com -d amazonaws.com

3. Specify minimum size of image to be downloaded (width x height):

   imagebot http://website.com -s 300x300

4. Stay under http://website.com/albums/new:

   imagebot http://website.com/albums/new -u http://website.com/albums/new

5. Launch monitor windows for live images:

   imagebot http://website.com -m

6. Set user-agent:

   imagebot http://website.com -a "my_imagebot(http://mysite.com)"

7. For more options, get help:

   imagebot -h

Dependencies

1. python-gi (Python GObject Introspection API)

   On Ubuntu:

   apt-get install python-gi

2. scrapy (a powerful web crawling framework)

   It will be automatically installed by pip.

3. Pillow (Python Imaging Library)

   It will be automatically installed by pip.

Download

• PyPI: http://pypi.python.org/pypi/imagebot/

• Source: https://bitbucket.org/amol9/imagebot/