Hello,
Thank you for contacting Rank Math and sorry for any inconvenience that might have been caused due to that.
Your robots.txt is loading fine, and there is no such directive in the robots.txt that might have stopped Googlebot from crawling your website.
Also, I checked your page source and there is no noindex tag present which means that your site can be crawled and indexed by any search engine.
You can revert your old site backup. Have you recently made a backup of your website? Some hosting services offer auto-backup solutions these days, so you might want to ask your hosting provider about this issue.
Can you please share a screenshot of the problem you are facing so we can assist you further? You can use a tool like ImgBB to capture screenshots. Similarly, you can record a screencast with a tool like Loom.
Looking forward to helping you. Thank you.
OK thanks.
Isn’t the ideal robots.txt file for wordpress already as we use it now?
Hello,
Rank Math by default includes the following rules to the robots.txt file. If you ever happen to remove these rules from your robots.txt but later want to include them by going to WordPress Dashboard > Rank Math > General Settings > Edit robots.txt, you can copy-paste the following rules. Make sure to replace yoursite.com with your domain name.
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: yoursite.com/sitemap_index.xml
Hope this helps you. Please don’t hesitate to contact us if you need further assistance.
Thank you.
Hello,
Glad that helped.
If you do have another question in the future, please feel free to create a new forum topic, and it will be our pleasure to assist you again.
Thank you.
