← All Tools
Robots Txt Generator🔒 100% local
Generate robots.txt. Privacy-first, no server uploads, no tracking.
Related Tools
Reads a rough list of robots.txt directives and re-emits them in a fixed order: a wildcard user-agent line, then every Disallow, then every Allow, then a Sitemap line. Directives you paste are normalised one per line and recognised only when they start with a known keyword. The rebuild happens locally; the file you paste is not uploaded.
How to use it
- Paste the directives you want, one per line, for example User-agent: * then Disallow: /private/.
- Press Process. The output box shows the rebuilt file.
- Copy the result into your public directory as robots.txt.
Worked example
User-agent: *
Disallow: /private/
Allow: /private/public/
Sitemap: https://example.com/sitemap.xmlbecomesUser-agent: *
Disallow: /private/
Allow: /private/public/
Sitemap: https://example.com/sitemap.xmlVerified in Node against the tool's parser: everything after the first colon is kept, so a sitemap URL survives whole.
Disallow: /admin/
Disallow: /tmp/becomesUser-agent: *
Disallow: /admin/
Disallow: /tmp/
# Sitemap: https://yoursite.com/sitemap.xmlWith no Sitemap line given, the fallback comment placeholder is used.
When to use it
- Drafting a robots.txt for a new site where the only goal is blocking a few directories.
- Checking how a set of directives reads when Disallow lines are grouped ahead of Allow lines rather than interleaved.
- Producing a starting file to hand-edit, when you know the parser is simplistic and will check the result yourself.
What it does not do
- The Sitemap line is copied as written, with no check that the URL resolves or that the domain is yours. A typo goes into the published file unchanged.
- Every user-agent you supply is discarded. The output always begins with 'User-agent: *', so per-bot rules are collapsed into one group.
- Crawl-delay is recognised during parsing but then filtered out of the output, and any line that is not a recognised keyword is silently dropped.
- There is no validation at all. It will emit a sitemap comment pointing at yoursite.com even if that is not your domain.