SEO Tool
Robots.txt Generator
Create a clean robots.txt file to control how search engine crawlers access sections of your website.
Use * to target all crawlers.
Enter one path per line.
Generated robots.txt
User-agent: * Allow: /
Robots.txt controls crawler access, but it should not be used to protect sensitive information.
How to use this tool
Choose the crawler you want to target, define allowed and blocked paths, then optionally add your sitemap URL. Copy the generated content and save it as robots.txt in the root directory of your website.