Google not indexing pages from sitemap_index.html

#533626
  • Resolved Dclempa
    Rank Math free

    Google will only index pages directly from the page-sitemap.xml and not the sitemap_index.xml

    Google search console will not make the jump through the first page to get the the actual sitemap to index the website pages.

    This creates a problem because the Robots.txt file uses the URL associated with sitemap_index.xml and not the actual sitemap itself which is page-sitemap.xml. Can you explain what is going on here??? I would greatly appreciated it. I am trying your free version because I want to buy the PRO if it works

    Thanks——Dennis

Viewing 6 replies - 1 through 6 (of 6 total)
  • Nigel
    Rank Math business

    Hello,

    Thank you for contacting Rank Math for help with your content in your page-sitemap.xml not being indexed, and sorry for the inconvenience this has caused you.

    Please go through our guide to troubleshoot and fix any issues that may affect your page-sitemap.xml. If none of the fixes offered in the guide works, please may you share your website URL in the designated sensitive data area so we can check what may be the issue.

    Hope that helps. Please let us know if you have questions.

    I am not sure you understand what I am referring to. I can submit the Page-sitemap.xml file to google and there is nothing wrong with the actual page-sitemap.xml. The issue is that your plugin creates another page sitemap_index.xml that it puts into the robots.txt file. If Google accesses the sitemap_index.xml file upon crawl, it will not jump through the link to the page-sitemap.xml which is the actual sitemap. The issue revolves around your plugin creating the sitemap_index.xml and putting that into the robots.txt file instead of putting the actual sitemap itself (page-sitemap.xml) into the robots.txt file. There is nothing I can troubleshoot because I cannot stop your plugin from creating a link to the sitemap page inside the robots.txt file rather than the sitemap itself. Maybe you can explain why there needs to be two separate pages created by your plugin regarding the sitemap (sitemap_index.xml and page-sitemap.xml) instead of a single page for the sitemap??? Website url is https://trident-sa.com

    Thanks for you assistance

    Nigel
    Rank Math business

    Hello,

    I’m so sorry. I did not include the article link in my last reply: https://rankmath.com/kb/fix-sitemap-issues/

    I checked your sitemap_index.xml and there were no issues with it. If Google is not crawling your sitemap_index.xml file, please try submitting this alternative sitemap index URL: https://trident-sa.com/?sitemap=1. You can follow this guide for how to manually submit a sitemap to Google: https://rankmath.com/kb/submit-sitemap-to-google/

    If you would like to edit the robots.txt and replace the sitemap_index.xml with page-sitemap.xml instead, please follow this guide for how to edit your robots.txt: https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/

    Hope that helps. Please let us know if you have questions.

    Hello,

    It seems you’re referring why there’s a /sitemap_index.xml page and why Rank Math adding /sitemap_index.xml path to your robots.txt file.

    Google is smart enough to jump from URL to URL. Means, Google will be able to crawl any URLs/child-sitemap (like /page-sitemap.xml , /post-sitemap.xml) under /sitemap_index.xml.

    Since you don’t have any posts on your site, no post-sitemap.xml has been generated automatically.

    However, as there can be several child sitemap for a site, sitemap_index.xml is used as a mother sitemap that holds/contains all other sitemaps of your site. So you don’t have to submit each sitemap manually to GSC.

    Here you can check Rank Math’s sitemap as an example: https://rankmath.com/sitemap_index.xml

    You can simply submit sitemap_index.xml to GSC or page-sitemap.xml there. There’s no restrictions on that. And also, you can edit robots.txt file and edit the sitemap URL there as my colleague mentioned above.

    Hope that clarifies your doubt.

    Thank you.

    Thanks for the reply and yes I know I can manually edit the robots.txt file and change the sitemap link, however, I want to bring to your attention that in the Google Search console, Contrary to what you state about Google being smart enough, it will NOT make this jump when manually submitting the sitemap (sitemap_index.xml) page.Perhaps Google will make that jump when the crawlers hit the website, however, it definitely will not do it manually. The only way I can submit the sitemap to Google is by using the actual Page-sitemap.xml. I think this is an issue that you guys might want to test yourself and address. My concern is that if I do not edit the Robots file as you suggested in your response, then the Crawlers might ignore the sitemap as well. I am not sure if this is a recent Google change or not, but worth investigating further. Thanks for your help—-Dennis

    Hello,

    The method we use to generate our sitemaps is perfectly acceptable and is mentioned in the Google documentation here: https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps

    This is the best method to generate large sitemaps instead of including all the links on a single page.

    Your website doesn’t have a lot of pages so having the links in the main sitemap wouldn’t hurt but we need to cater to the broader audience and a lot of our users have thousands of pages so they would benefit from this method.

    Having said that, even for smaller websites, the method we use is perfectly compliant with the guidelines we shared.

    Don’t hesitate to get in touch if you have any other questions.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 6 replies - 1 through 6 (of 6 total)

The ticket ‘Google not indexing pages from sitemap_index.html’ is closed to new replies.