rethinkdb python library
Distributed web crawling with browsers
WARC writing MITM HTTP/S proxy
url canonicalization library for python and java
Rudimentary rethinkdb python library with some smarts, perhaps some dumbs
Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
Sort-friendly URI Reordering Transform (SURT) python package.