dripfeed 0.9.0

Create an RSS feed of a webcomic archive, for slow perusal.

Create an RSS feed of a webcomic archive, for slow catchup.

Ever time I discover a new webcomic that’s worth following, I lose hours (often night-time hours) to catching up with the archives. This tool exists to avoid this problem: I can create my own dripfeed for the comic, schedule it with cron to update two or three times a day, and add the feed to my ordinary feed reader. So long as dripfeed updates more often than the comic author, my dripfeed will catch up eventually, and I can switch to the official feed from then on.

Example usage

Create the feed:

dripfeed init gunnerkrigg  # name for dripfeed commands like "update", "remove" (commandline-friendly)
              --rss ./gunnerkrigg.rss  # rss file for output (will be created)
              --url 'http://gunnerkrigg.com/?p=1'  # where to find the first page
              --next "//img[@src='http://www.gunnerkrigg.com/images/next_a.jpg']/.."  # XPath for "next" link
              --name 'Gunnerkrigg Court'  # optional long name for output (doesn't have to be commandline-friendly)

The --next parameter is an XPath expression that extracts the <a> element whose href points to the next page. (This expression will be used for all pages of the comic.)

This places configuration for gunnerkrigg in a config file at ~/.dripfeed.cfg (creating the file if it doesn’t already exist).

Now running:

dripfeed update gunnerkrigg

will update the rss feed at ./gunnerkrigg.rss and store progress in ~/.dripfeed.cfg: I’d expect this command to go in a cron job.

At present success generates output on stdout, so you’ll want to redirect stdout to /dev/null if you run this from cron; I intend to change this in a later version. Errors are recorded in the RSS feed, and you can run dripfeed update with a --debug flag to see a full stack trace of the error.

Output

The RSS feed entries are intentionally very very simple: they contain just a link to the page, and some placeholder text telling you which episode you’re looking at (counting from episode 1 at the initial URL).

It would be possible to extend the tool to include some degree of content scraping: more XPath expressions could optionally extract the comic image, title, commentary, etc. I do not intend to do this; of course you’re welcome to fork the code and make whatever changes you like, but I will not accept pull requests adding these features. The reason is that I want you to visit the original comic pages: making a living from webcomics is tricky enough as it is, and many comics are either directly or indirectly ad-supported. This script is not a syndication tool and is emphatically not intended to make business any harder for the authors whose work I admire.

Requirements

• Python >= 2.6

License

MIT licensed. See the bundled LICENSE file for more details.

TODO

Not sure when I’ll get around to these, but here are a few things I would like to do with it (several of them more for the learning experience than because the task really demands it):

• python3 support

• switch print() output to use logging

• --quiet and --verbose options (and suitable defaults: update should be quiet since it is intended to be run under cron)

• Interactive init that prompts for necessary args and validates them (especially the xpath).