Last released Oct 2, 2026
Crawl websites within their robots.txt (RFC 9309), with a recorded reason for every request.