Hello,
I’m sorry to hear about the issue you are facing with your robots.txt file causing a 403 error. To resolve this problem, you can generate and edit your robots.txt file using the Rank Math SEO plugin.
You can edit your robots.txt file with Rank Math by navigating to Rank Math SEO → General Settings → Edit robots.txt. If you encounter any issues while editing the file, there might be a physical robots.txt file present in the root folder of your website or another plugin creating the file.
Here are some common reasons and solutions:
- Presence of a Physical File: If there is a physical robots.txt file in your site’s root folder, you need to remove it via FTP or cPanel File Manager.
- Other Plugin Creates Robots.txt: Disable any other plugin that might be creating the robots.txt file.
- Permission Issue on the Web Server: If there are permission issues on the web server, contact your web host for further assistance.
If you need further guidance or encounter any difficulties, please let us know. Thank you for bringing this to our attention.
Sources:
I Can’t Edit My Website’s Robots.txt » Rank Math
Common robots.txt Issues & How to Fix Them » Rank Math
How to Fix the ‘Blocked Due to Other 4xx Issue’ Error » Rank Math
Hello,
We’ve checked your robots.txt and it seems to be working fine.
This can happen when you use a content delivery network (CDN) like Cloudflare. Rank Math does not create a physical robots.txt file on your site. Instead, the file is created and displayed when a user visits your robots.txt URL.
In this case, we recommended creating a robots.txt file in your website’s root directory and copy the same rules you have set in Rank Math.
Let us know how this goes.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
