HyperCloudHostHyperCloudHost
Register
WEBSITE & SEO

Robots.txt Generator

Specify which parts of a website compliant crawlers may visit. Review the generated text before installing it on your domain.

Your input and results stay on your device. Only the tool name is sent to check the shared download limit.

01

Set your options

Settings robots-txt
Crawler
Path fields apply to Custom paths. Supports * and $.
02

Your result

Local

Your result will appear here.

Set your options, then choose Generate.
Downloads: 10 per IP in 5 minutes, shared across tools. Repeated excess requests increase the waiting period.

How to use

  1. Select crawlers and the starting policy.
  2. Enter one path per line and an optional sitemap URL.
  3. Generate, review, and upload the file to your domain root.
Where should I put robots.txt?

Place it at the domain root, such as https://example.com/robots.txt. This tool downloads a file and does not overwrite this website’s robots.txt.

More tools for your work

Hash Generator

Calculate MD5 and SHA hashes from text.

Open tool

Password Generator

Generate random passwords with your chosen options.

Open tool

Lorem Ipsum Generator

Placeholder text for layouts and mockups.

Open tool