Hello,
I understand that you are facing an issue where Google Search Console indicates that your URL is not available to Google, even though your robots.txt file is correct in Rank Math and on your site. It seems like you have taken some steps to troubleshoot the problem by deactivating plugins, disabling the CDN, and waiting for 12 hours.
Given that you have many pages set to "noindex," this could potentially be a reason why Google is reporting this message. When pages are set to "noindex," they won’t appear in search results, but Google may still try to access them.
To further investigate and resolve this issue, I recommend the following steps:
-
Review the Status of the URL: Check if the URL is blocked or allowed by robots.txt. You can use tools like the robots.txt testing tool to ensure that Googlebot can access your site properly. Here is an example of how to test your robots.txt file:
- Open the robots.txt testing tool.
- Enter your website’s URL for testing.
- Select the user agent (Googlebot).
- Click the TEST button to validate your robots.txt file.
-
Export the Report from Google Search Console: If you have many URLs under this warning, consider exporting the report from Google Search Console to analyze it further. You can export the URLs and review them to determine if any specific pages are causing issues.
If after performing these steps you still encounter difficulties, please reply to this ticket, and a human agent from the Rank Math Support team will assist you further.
Thank you for your patience and cooperation in resolving this matter.
Sources:
No Information is Available for This Page » Rank Math
How to Fix ‘Blocked by robots.txt’ Error in Google Search Console » Rank Math
How to Fix the "Indexed, though blocked by robots.txt" Error » Rank Math
Hello,
Thank you for your query and we are so sorry about the trouble this must have caused.
We checked your website and indeed the robots.txt on your website is working fine with the correct directives.
We also checked the website directly on Google search and the site is currently indexed:
In this case, Google may currently be showing an outdated report. Please give it some time for Google to recrawl your website and update the GSC reports.
Looking forward to helping you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
