Hello,
Thank you for contacting Rank Math, and sorry for any inconvenience caused.
Google does not honor noindex in robots.txt file.
You can read more about this change here: https://www.deepcrawl.com/blog/best-practice/robots-txt-noindex-the-best-kept-secret-in-seo/
I hope this helps. Looking forward to helping you.
how to block indexing of directory files
Hello,
Thank you for following up.
Please use Disallow: then add the directory or page, relative to the root domain, that should not be crawled by the user agent.
You can also check Google’s recommendation for creating a robots.txt file:
https://developers.google.com/search/docs/advanced/robots/create-robots-txt?hl=en
I hope that helps. Thank you, and please don’t hesitate to contact us if you need further assistance.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
