-
A robots.txt file is currently blocking the complete indexing of our website by Google. We have used the robot.txt tester and validator to edit it, but the file is still blocking indexing. We’ve reviewed your troubleshooting documentation, but still cannot find out where the issue lies. Is it please possible for you to assist? Thank you!
-
Hello,
We apologize for the issue you are facing with editing your robots.txt file and the indexing of your website by Google. It sounds like you’ve already taken some important steps by using the robots.txt tester and validator, which is great.
To assist you further, please ensure that you are following these steps to edit your robots.txt file in Rank Math:
-
Navigate to your robots.txt file in Rank Math, which is located under WordPress Dashboard → Rank Math SEO → General Settings → Edit robots.txt.
-
In the code editor available in the middle of your screen, paste the updated code you’ve copied from the robots.txt Tester and then click the Save Changes button to reflect the changes.
Please note that any changes made in the tester will not automatically reflect on your website’s robots.txt unless you manually update it as described above.
If you continue to experience issues, could you please reply to this ticket with the URL of your site? This will help us investigate further and provide more specific assistance.
Thank you for raising this concern, and we look forward to helping you resolve it!
I have followed the solutions provided by the AI agent, and my robots.txt file is not showing the updates on the server.
Hello,
Thank you for your query and we are so sorry about the trouble this must have caused.
If you can’t edit the robots.txt, please check your website’s root directory files for a physical robots.txt file there and remove it.
Also, please check with your hosting provider and ensure full permission for the modification of your robots.txt file.
If the issue persists, we might need to take a closer look at the settings. Please edit the first post on this ticket and include your WordPress & FTP logins in the designated Sensitive Data section.
Please do take a complete backup of your website before sharing the information with us.
It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:
https://wordpress.org/plugins/temporary-login-without-password/
You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):
https://wordpress.org/plugins/wp-security-audit-log/
We really look forward to helping you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hello,
We’ve checked your file manager, but there’s no robots.txt file.
In this case, please check with your web host for the plug-ins that are automatically activated on your site under /mu-plugins.
The robots.txt must be handled by one of the features or apps from your hosting.
Let us know how this goes.
I just checked with GoDaddy and they are claiming the issue is with the RankMath SEO plugin. “The only robots.txt files you have are related to this plugin seo by rank math. So
there is no other robots.txt files.”They are showing a robots-txt directory in our WordPress File Manager related to RankMath. I have a screenshot, but there does not seem to be a way to attach it here.
Hello,
You can upload screenshots using this tool and add the link here.
Looking forward to helping you.
We are checking in because we sent the screenshots as requested by Reinelle about 5 days ago and have not received a response. In the meantime, we are getting pressed by our client asking why his site is back to being indexed by Google yet.
Thank you so much for your assistance. (If we have to deactivate RankMath, (1) will that delete the robots.txt file that’s blocking indexing and, (2) will our RankMath SEO work be retained on the site’s webpages?)
Hello,
Our plugin is not generating the robots.txt file on your site. Some other plugin or your server is overriding our plugin’s file. Please disable plugins one by one to figure out which plugin is adding the robots.txt. If it isn’t coming from another plugin, you should check with your host, as they will be in a better position to help you fix it.
Please do not hesitate to let us know if you need our assistance with anything else.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
-
The ticket ‘Unable to edit our robot.txt file’ is closed to new replies.