Hello,
Thanks for contacting us and sorry for any inconvenience that might have been caused due to that.
Can you please follow these steps?
1. Flush the Sitemap cache by following this video screencast:
https://i.rankmath.com/pipRDp
2. Exclude the Sitemap files of the Rank Math plugin in your caching plugin. The cache could be via a plugin or from the server. For plugins or Cloudflare, please follow this article:
https://rankmath.com/kb/exclude-sitemaps-from-caching/
That should fix the issue. Please let me know if that does not. We are here to assist.
HI, I have followed the steps and submitted to google again. How do I check if the sitemap is exclued from being cached?
Hello,
You can use a tool like httpstatus.io to check that.
You can also share the URL of your sitemap and we will check it for you.
Looking forward to helping you.
Thank you.
Hello,
I have updated the sitemap as requested. Can you please check further?
Thank you.
Hello,
There’s something currently on your website blocking the access of Googlebot to your sitemap because when we try to get the sitemap via the Googlebot we get a 403 error.
Please check your robots.txt file and make sure it’s the default one to let Google crawl your sitemap correctly. Currently, that file doesn’t even have the appropriate entry to the sitemap in there.
Here’s how you can edit that file with Rank Math: https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/
Also, make sure that you don’t have any rules in Cloudflare or any other firewall blocking the crawling from Google to your website.
Hope this helps solve your issue.
Don’t hesitate to get in touch if you require any further assistance.
Thank you.
Hi,
I have checked my robots.txt
It should work for googlebot
User-Agent: Googlebot
Allow: /*.css$
Allow: /*.js$
Hello,
It seems you are not using the format of Rank Math default robots.txt file.
https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/#default-rules
Please take a backup of your existing robots.txt file’s rules and replace it with Rank Math’s default robots.txt file’s rules and see the error persists or not.
Hope this helps. Looking forward to helping you.
Thank you.
Hi,
Do I have to remove all the existing rule and replace them with the default rule. What if I want to keep them? THank you.
Hello,
We recommend using the default robots.txt, but if you want to keep the existing rules, just add the sitemap URL in the robots.txt below the rules.
Sitemap: yoursite.com/sitemap_index.xml
I hope this helps.
Looking forward to helping you.
Hi,
I have the sitemap URL added to the robot.txt already. Your colleague said there is a 403 error – something blocking the googlebot. Does having the sitemap URL help resolve the problem?
Hello,
I checked your robots.txt file and noticed that you added your sitemap URL at the top of your robots.txt file. Please add your sitemap URL below your robots.txt file as my colleague instructed here.
Hope that helps. Looking forward to helping you.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
