No syntax check of robots.txt #11
Description
If we try to parse an invalid robots.txt (containing just garbage), parse still returns ok (with an empty set of rules).
{{ message }}
If we try to parse an invalid robots.txt (containing just garbage), parse still returns ok (with an empty set of rules).