robots.txt generator

Generator a Robots.txt file to let Googlebot and other User-Agents know which parts of a domain can be crawled by a robot.

Add Directive

Sitemaps