FREE ONLINE MARKETING TOOL
Robots.txt Generator.
Generate basic crawler directives and a sitemap reference for your site.
No signupRuns in your browser
YOUR WORKSPACE
Start working.
Enter your details, then click the action button to see your result.
YOUR RESULTReady for inputs
—
Fill in the fields and choose Analyze now.
PRACTICAL GUIDE
How to create a robots.txt file
Specify a user agent, allowed or disallowed URL paths, and an optional sitemap URL. Place the reviewed text at your website root as robots.txt. Test critical rules before publishing.
Does robots.txt stop a page appearing in Google?
Not reliably. Robots.txt guides crawling; it does not secure confidential information or guarantee that a known URL will not appear in search results.
