Query For Robot.txt

Hello,

We apologize for the issue you are facing with your robots.txt file. If the file is missing from your cPanel, you can indeed add it manually. Rank Math automatically generates a robots.txt file, but if you prefer to manage it manually, you can create a new file in the root directory of your website.

How to Add robots.txt in cPanel:

1. Log into cPanel and navigate to File Manager.
2. Go to the root directory of your WordPress site.
3. Click on + File to create a new file and name it robots.txt.
4. Edit the file by right-clicking on it and selecting Edit. You can then add your desired directives.

You do not need to disable the Rank Math plugin to add or edit the robots.txt file manually. However, if you want Rank Math to manage it, ensure that there is no other robots.txt file present in the root directory that could conflict with it.

Regarding Permanent URL Removal from Google:

To remove unwanted URLs permanently from Google, you can use the following methods:

• Google Search Console: You can submit a removal request for URLs, but as you mentioned, this is temporary and lasts for about six months. To keep a PDF or any URL permanently out of Google’s search index, you should block search engines from crawling it using the robots.txt file. This way, even if someone links to it, Google will not crawl or index it.

For more detailed instructions on how to edit your robots.txt file with Rank Math, you can refer to this guide.

If you have any further questions or need assistance, please feel free to reply to this ticket, and a human agent from our support team will assist you further. Thank you for raising your concerns!