-
Are there free alternative tools to https://ahrefs.com/it/robot? This tool finds problems with Ahrefs crawlers, but I’m interested in knowing if Google, Bing, and Yandex crawlers are fine.
I used the official tools from Google, Bing and Yandex for robots.txt and Sitemap, but it is clear that I have problems with the crawlers, even if the results are fine.
-
Hello, I’m sorry to hear that you’re experiencing issues with the crawlers. While Ahrefs is a great tool for identifying problems with their crawlers, there are free alternative tools available to check how Google, Bing, and Yandex crawlers interact with your site.
For Google, you can use the Robots Testing Tool to check your robots.txt file and ensure that there are no directives blocking Googlebot from accessing your site. Here’s an example of what it would look like:
For Bing and Yandex, you can use their respective webmaster tools to check for similar issues with their crawlers.
If you’re still experiencing problems after using these tools, please feel free to reply to this ticket, and a human agent from the Rank Math Support team will assist you further.
Sources:
Common robots.txt Issues & How to Fix Them » Rank Math
How to Fix ‘Blocked by robots.txt’ Error in Google Search Console » Rank Math
How to Fix the “Indexed, though blocked by robots.txt” Error » Rank Math
Using Rank Math’s ‘Quick Actions’ Admin Menu Item
The ticket ‘free alternative tools for crawlers’ is closed to new replies.