Hello,
I’m sorry to hear that you’re facing an issue with your robots.txt file not being accessible on your website http://www.keto-programs.com. To fix this, you can follow these steps:
-
Check the robots.txt File: Ensure that your robots.txt file is correctly configured and accessible. You can use the Robots Testing Tool to verify the content of your robots.txt file.
-
CDN Consideration: If you are using a content delivery network (CDN) like Cloudflare, sometimes it can cause issues with the robots.txt file. You can add the following rule to your robots.txt file and test it again in PageSpeed Insights:
Disallow: /cdn-cgi/
If you encounter any difficulties or need further assistance, feel free to reply here, and a human agent from the Rank Math Support team will assist you promptly.
Thank you for reaching out!
Best regards.
Sources:
Common robots.txt Issues & How to Fix Them » Rank Math
How to Fix ‘Blocked by robots.txt’ Error in Google Search Console » Rank Math
No Information is Available for This Page » Rank Math
Hello,
Thank you for contacting us and sorry for any inconvenience that might have been caused due to that.
We checked your robots.txt file and we can see an invalid line appearing there from the WP-Optimize plugin you’re using.
To determine that, please follow this troubleshooting guide:
https://rankmath.com/kb/check-plugin-conflicts/
Here is a video you can follow as well:
https://www.youtube.com/watch?v=FcIWRhcUP2c
Only the site admin would know and your users won’t be affected by this troubleshooting.
Let us know how it goes. Looking forward to helping you.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
