Hello,
Thank you for contacting Rank Math, and sorry for any inconvenience caused.
I checked the page source of the product shared in the sensitive data section, it is already set to index.
The robots.txt file and sitemap are also working fine.
Please note that crawling and indexing of the website depend on the authority and crawl budget of the website.
It is completely upto Google if they want to index or not index your website. So, if you haven’t willingly or accidentally made your products noindex and Google still isn’t indexing your products – it means that either Google doesn’t want to index or hasn’t crawled your products yet because it hasn’t allocated that much crawl budget to your website.
Check this website for more details.
Please make sure that you have submitted the sitemap in the Search Console:
https://rankmath.com/kb/submit-sitemap-to-google/
Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.
Hello,
want to share an update.
“It is completely up to Google if they want to index or not index your website.”
Well, this statement is wrong.
it’s related to the “Links per sitemap” number. By default is 200 links, and when a woocommerce store has 50 000+ products or more, or thousands, 200 links per sitemap is a very small number, that generates lots and lots of sitemaps, so Google search console cannot process a bunch of sitemaps in our case it was more than 300 sitemaps.
Checked other marketplace who have millions of products, they have 40,000 products in one sitemap.
After changing 200 to the 10 000 links per sitemap, situation has changed.
Google search console found +30K new URL and started indexing them
I think you should document our situation, because your documentation says:
Earlier, the default value for this setting was 1000. But, we’ve observed that Google responds better to smaller pages in sitemaps. Thus, we’ve reduced the value to 200, which is our recommended setting.
It’s wrong. Maybe it works for small sites, but not for big-e-commerce sites.
thanks and hoping it helps for others.
Hello,
The documentation and recommendations are for an average WP website. Normally, Google responds better to small sitemaps but if you have a lot of sitemaps in your main sitemap then it can cause sometimes cause issues with the crawling.
Thanks for sharing the feedback with us. I have forwarded this to our team for further discussion.
Please don’t hesitate to get in touch in case you need any other assistance.
Thanks.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
