Hello,
Thanks for contacting us and we regret the inconvenience caused.
I visited your website and I noticed that your robots.txt has already blocked wp-admin.
“Indexed, though blocked by robots.txt” indicates that Google indexed URLs even though your robots.txt file blocked them.
Google has marked these URLs as “Valid with warning” because they’re unsure whether you want to have these URLs indexed.
Please make sure to deindex in your Search Console every URL in wp-admin except admin-ajax.php (or even that URL, since you don’t want it indexed, only accessible to robots).
Please refer to this tutorial for fixing the issue: https://ahrefs.com/blog/indexed-though-blocked-by-robots-txt/
After performing the steps in the above tutorial, please give Google some time to re-crawl your website and update the changes.
Hope this helps.
Thanks.
Thank you much. I’m sorry but still struggling to understand what are the exact entries I need to enter in the robots.txt
Can you please share the sample code?
Hello,
I have checked the robots.txt of your website and it is already correct.
You do not need to change anything in the robots.txt file.
As my colleague advised, please wait for some time and let Google crawl your website.
I hope this helps. Looking forward to helping you.
Thank you for the clarification. Ready to wait but I mean I have not changed this robots.txt file for ages and that is the reason for my concern what should I do to eliminate the issue.
The article mentioned above gives a lot more details on how to do this and that ….
Hello,
Thanks for your reply.
I suggest you clear your website’s cache since Google might be seeing a cached version of your website. You may follow this tutorial to clear the cache of your website: https://www.wpbeginner.com/beginners-guide/how-to-clear-your-cache-in-wordpress/
Further, it would just be a waiting game until Google crawls your website again and de-indexes the nav-menu.php page.
Hope this helps.
Thanks.
please make sure to deindex in your Search Console every URL in wp-admin except admin-ajax.php (or even that URL, since you don’t want it indexed, only accessible to robots).
I don’t understand how to achieve this part. One of your colleagues mentioned that I don’t have to do anything. But it seems I have to do something.
BTW I have a scheduled purge of Cache already in place
Hello,
Thank you for the follow-up, and sorry for the delay.
My colleague mentioned that to follow the correct robots.txt format, which you are already using.
In this case, you can also resubmit your main sitemap /sitemap_index.xml to your Google Search Console and give it some time to recrawl your website.
Once Google recrawls, it should only reach your website pages in your sitemap and not your WordPress files.
I hope that helps. Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
