Hello,
Thank you for contacting Rank Math and bringing your concern to our attention. I’m sorry for any inconvenience this issue may have caused you.
I’ve tried checking your site, but it seems that the user has limited access to your site. Please update it to an admin-level to check further.
For this issue, please also check your Nginx config file and ensure there are no directives blocking robots.txt on your site.
You can also check this link and let us know if the issue persists.
I hope that helps.
Thank you, and looking forward to helping you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hi Reinelle,
I’ve amended the access so you will have sufficient permission now through the same temp access link.
I’ve also included the etc/nginx/nginx.conf file and it does not include anything to do with robots.txt
In addition to this, I’ve also searched for robots.txt from the root directory and pasted the results in the sensitive data section for your perusal if required.
Hope this helps.
Hello,
Thank you for that information.
I’ve checked your site, and it seems that the robots.txt option is not locked.
Have you tried creating a robots.txt file in your website’s root directory? If not, please do that and check if that makes a difference.
Meanwhile, I’ve already raised this to our dev team for further checking. We will get back to you shortly.
Looking forward to helping you.
Thank you.
Hi Reinelle,
Thanks for your help so far.
Just to let you know I’ve even rebooted the website and unfortunately, this hasn’t resolved the issue.
I have created the robots.txt file on the root of the website directory /var/www/example.com/htdocs/robots.txt and this has worked successfully.
If RankMath requires to update this file dynamically, it’ll be unable to do so at present, due to it being manually created, is that correct?
I’m still curious as to why RankMath is unable to generate/write robots.txt as designed.
Hello,
Thank you for getting back to us.
Just checked your site and now it seems to have the robots.txt file since you have uploaded it in the root directory.
One of the common reasons you can not edit the robots.txt file is if there is a physical robots.txt file is present in the root folder of your website as you have said.
Here is a doc on more on this: https://rankmath.com/kb/cant-edit-robots-txt/
The robot file that Rank Math generates is virtual and it can only do so when there is no file in your root directory.
If this was not happening then it would be worth investigating this from your server end and backend. Seems like you have already removed your FTP credentials. If you want to proceed with the one from the root folder it is fine since it’s one and the same thing.
We are looking forward to helping you.
Thank you.
Thanks for your help with this Brian.
I think we’ll leave it there and I’ll just go with this solution now that the robots.txt file is there on the root of the website directory and RankMath is reading it correctly.
One thing that did cross my mind is that Yoast SEO was installed prior to RankMath but Yoast was completely removed prior to RankMath’s installation. Whether Yoast creates or alters the permissions of the website directory, I’m not sure but there definitely wasn’t anything on the root directory.
Thanks again for all your help and prompt responses to my problem!
Hello,
Thank you for the update.
We are glad it worked for you, and it seems that it is already resolved. Could you please confirm?
Looking forward to helping you.
Thank you.
Hello,
We are super happy that this issue is already resolved. If you have any other questions in the future, know that we are here to help you.
If you don’t mind us asking, could you please leave us a review (if you haven’t already) on https://wordpress.org/support/plugin/seo-by-rank-math/reviews/#new-post about your overall experience with Rank Math? We appreciate your time and patience.
If you have another question in the future, please feel free to create a new forum topic, and it will be our pleasure to assist you again.
Thank you.
