Last released Jul 22, 2024
Provides python access to Googles parser for robot.txt files as used by their GoogleBot webscraper.
Supported by