Robots.txt Generator
Build a correct robots.txt with allow and disallow rules for common bots, ready to copy.
Free, no signup needed. Runs 100% in your browser.
Loading…
How to use this tool
- Choose which bots to allow or block.
- Add paths to disallow.
- Copy the file to your site root.
Frequently asked questions
Where does robots.txt go?
At your site root, for example yoursite.com/robots.txt, served as plain text.
Does blocking hide pages from everyone?
No. It only asks polite crawlers to stay out. Do not use it for secrets.
Can I add a sitemap line?
Yes. A sitemap entry at the end points crawlers to your URLs and is widely supported, unlike crawl-delay, which major engines ignore.
Related tools
Need a website for this?
Describe your idea once and BYOB plans, builds, and deploys your site.
Embed this tool
Paste this snippet to show this tool on your own site.