database of robots.txt-derived permissions
