robots.txt builder
Quick help
Use one literal path per line to disallow crawling for all agents. Enter NONE to allow all. Paths may contain letters, digits, slash, hyphen, underscore and dot only. This does not restrict access or guarantee removal from search. Review before placing at the site root.
- Enter your details and replace fictional examples.
- Prepare and review the result.
- Download, print or export a draft to keep your work.
What it does not do: No sending, hosting, publishing, legal review or automatic reminders.
Use one literal path per line to disallow crawling for all agents. Enter NONE to allow all. Paths may contain letters, digits, slash, hyphen, underscore and dot only. This does not restrict access or guarantee removal from search. Review before placing at the site root.
Format references and last checked dates: source notes. Generated files need review in your destination app or host.
Nothing is fetched or sent. Previews show up to 100 CSV rows; downloads include all rows, up to 10,000. CSV downloads prefix possible spreadsheet formulas with an apostrophe. Drafts preserve original text. Keep exported personal records private.
Import is kept in this page's memory. The original profile file is unchanged.
Preview / results
Enter your details on the left to prepare your result.