Hello,
Thanks for contacting us and sorry for any inconvenience that might have been caused due to that.
“Indexed, though blocked by robots.txt” indicates that Google indexed URLs even though your robots.txt file blocked them.
Google has marked these URLs as “Valid with warning” because they’re unsure whether you want to have these URLs indexed.
Please refer to this tutorial for fixing the issue: https://ahrefs.com/blog/indexed-though-blocked-by-robots-txt/
After performing the steps in the above tutorial, please give Google some time to re-crawl your website and update the changes.
Hope this helps. Let us know if you need any further assistance.
please give me short solution I am unable to understand this ahref link content. It is so hard to understand. please please please help me
Hello,
To check and advise accordingly, please share the affected URL/s in the sensitive data section
It is completely secure, and only our support staff has access to that section.
Looking forward to helping you.
Thank you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hello,
The page you’ve shared is getting redirected too many times. Please see the screenshot in the sensitive data section of this ticket. Since you cannot redirect URLs with parameters using the redirection manager of Rank Math, could you confirm if you’ve created the redirection using some other plugin or the htaccess file?
However, I checked the robots.txt on your website and there are many rules that aren’t required here. For example, you’ve allowed the plugins directory and namely a plugin xt-visitor-counter to be crawled.
Also, you’ve added all the individual sitemaps in the robots.txt, you should only add the main sitemap here (i.e, your-domin.com/sitemap_index.xml).
Please double-check your robots.txt and keep only the rules that are required.
Hope this helps.
Thank you.
please send me a best robot txt file for my site. https://kotokisuojana.com
Hello,
We always recommend our default robots.txt for the websites which looks like this:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://yourdomain.com/sitemap_index.xml
You may need to remove all the sitemaps URLs and only include your main sitemap. For the other rules, you can double-check if you really need them on your site.
I hope that helps.
Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
