crawler_detection 0.1

A simple python script to identify and classify possible crawlers through analysis of web server log files

A simple python script to identify and classify possible crawlers through analysis of web server log files. The script categorizes identified web crawlers in to three as “known”, “suspicious” and “other”.This works with log files in Apache common log format.

Information about installation:

http://code.google.com/p/crawler-detection/