Hello,
Thank you for contacting Rank Math and bringing your concern to our attention. I’m sorry for any inconvenience this issue may have caused you.
There are various reasons for a page to be excluded from being indexed. Please check this link for more information:
https://rankmath.com/kb/crawled-currently-not-indexed/
You can also share your website so we can check for any issues. Please add it in the sensitive data section
It is completely secure, and only our support staff has access to that section.
I hope that helps.
Thank you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hello,
Crawled – currently not indexed simply means that the page was crawled but is yet to be indexed. There can be several reasons for this.
I suggest you double-check the points mentioned in this article on a few recently published pages that are affected here and wait for Google to crawl the website again and see if the pages are indexed:
https://moz.com/blog/crawled-currently-not-indexed-coverage-status
Discovered, not indexed, simply means that Google is aware of the page’s existence but the pages are yet to be crawled. Typically, Google wanted to crawl the URLs but this was expected to overload the site; therefore Google rescheduled the crawl. This is why the last crawl date is empty on the report.
I checked your website and was able to find posts that are not yet indexed. For example, the post /blog/sourcing-agent-101/. I checked the source of the page and it is already set to index.
I also accessed the page as Googlebot and there are no issues found.
Upon investigating, I found that the sitemap URL which you’ve added in the robots.txt is not the URL of Rank Math’s sitemap. Though the URL is redirecting to Rank Math’s sitemap, the bots have to go through an unnecessary redirection.
You can have a look at the screenshots in the sensitive data section for a quick reference.
Please replace the sitemap URL in the robots.txt with this URL: your-domin.com/sitemap_index.xml (please replace “your-domain.com” with your actual domain).
Once done, please clear your website and server’s cache after modifying the robots.txt wait for Google to crawl the website again and see if the posts start indexing.
Hope this helps.
Thank you.
User-agent: Googlebot
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://sourcingnova.com/sitemap_index.xml
Is it all right now?
I think the sitemap url was from yoast plugin.
Hello,
As you’re not defining any special rules, there is no need of adding the Googlebot User-agent statement.
You can replace the entire contents of the robots.txt with the below rules:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://sourcingnova.com/sitemap_index.xml
Hope this helps.
Thank you.
Siteground support team let me add it.
I’m glad to remove it again.
Hello,
Glad that you were able to replace your robots.txt rules.
Now, wait for Google to crawl the website again and see if the posts start indexing.
Please let us know if you have other questions to assist you further with anything else.
Looking forward to helping you.
Thank you.
3 posts got indexed today.
I believe updating sitemap url is the key.
It was caused by switching from yoast to your plugin.
Maybe you need to remind other users if they need to switch from yoast to rank math.
Hello,
Yoast has a similar sitemap structure as Rank Math. The index sitemap URL of Yoast and Rank Math is the same i.e, /sitemap_index.xml.
However, since the URL was not of Rank Math’s sitemaps, the bots had to go through an unnecessary redirection.
We’re glad that the posts are now getting indexed.
Please feel free to get in touch if you require any further assistance.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
