Sitemap pages return 404 and pages not indexed

Hello,

We apologize for the issues you are facing with your sitemap and indexing on Google. Thank you for providing detailed information about your situation.

Here are some steps you can take to troubleshoot and potentially resolve the 404 errors and indexing issues:

1. Flush Permalinks

Sometimes, flushing the permalinks can resolve 404 errors. To do this:

• Go to Settings → Permalinks in your WordPress admin area.
• Click the Save Changes button without making any changes.

2. Check for Plugin Conflicts

Since you mentioned that the sitemap keeps breaking, it might be due to a plugin conflict. We recommend using the Health Check plugin to identify any conflicts with other plugins.

3. Ensure Posts Are Set to Index

You mentioned that the Index checkbox is enabled, but please double-check that all your posts are set to index. You can do this by going to Rank Math SEO → Titles & Meta and ensuring that the Index checkbox is checked for all relevant post types.

4. Monitor Server Configuration

If you are using Nginx or Apache, ensure that your server configuration is correctly set up to handle the sitemap requests. Sometimes, server settings can inadvertently block access to the sitemap.

5. Check for Caching Issues

If you have caching enabled (either through a plugin or server-side), try clearing the cache. Caching can sometimes serve outdated versions of your sitemap, leading to 404 errors.

6. Review Robots.txt File Again

Even though you mentioned there are no blocks in your robots.txt file, it’s worth reviewing it again to ensure there are no unintended rules that could affect indexing.

7. Google Search Console

Since you are receiving conflicting messages in Google Search Console, ensure that you have submitted the correct sitemap URL and monitor any messages or warnings provided by Google regarding indexing issues.

If these steps do not resolve the issue, please reply to this ticket, and a human agent from the Rank Math Support team will take over and assist you further.

We appreciate your patience and understanding as we work to resolve this issue for you!

Sources:
Troubleshooting the “Couldn’t Fetch” Error in Google Search Console » Rank Math
Rank Math Sitemap Issues and Their Fixes » Rank Math

Hello,

Thank you for your query and we are so sorry about the trouble this must have caused.

We checked your sitemap and it seems to be working just fine.

To ensure that your sitemap is free from caching, make sure to follow this guide as well: https://rankmath.com/kb/exclude-sitemaps-from-caching/

As for the pages that are not indexed, please share some example URLs here so we can take a closer look.

Also, please submit the affected URL/s to your Google Search Console’s URL Inspection tool and share a full screenshot of the report.

Looking forward to helping you.

Hello,

Please update your robots.txt to our default and recommended rules for indexing:
https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/#default-rules

Also, you should remove the page-sitemap.xml from your GSC since you already submitted your sitemap index.

You can remove all of them and resubmit the /sitemap_index.xml only. Here’s our guide you can follow:
https://rankmath.com/kb/submit-sitemap-to-google/

Once done, give Google some time to crawl your URLs.

Looking forward to helping you.

Hello,

Please note that each time Google uses a sitemap to find a URL, the count of the Discovered URLs increases by one. Sitemaps is only one of the few methods Google uses to discover your URLs.

When the count of the Discovered URLs is zero or does not match your actual sitemap in your Google Search Console, it means Google didn’t use the sitemap to find the URLs.

It mostly happens when you have good internal linking or use the Instant Indexing plugin.

Here’s a link for more information:
https://rankmath.com/kb/zero-discovered-urls-through-sitemap/

Looking forward to helping you.

Hello,

To be sure that Google is crawling the website you can head over to Google Search Console and in the settings select the crawling report which can be found like so:

If you see stats for crawling it means that Google is seeing your website which is a good thing. If Google is seeing the website but not indexing, you need to head over to the indexing report on GSC and see if you are seeing any error preventing indexing mentioned there.

At the moment we can see that the sitemaps and the website are accessible to Googlebot so this could be just a matter of time or a content issue that prevents Google from indexing the website.

Don’t hesitate to get in touch if you have any other questions.

Hello,

Google uses its algorithm to determine if a webpage is suitable for its SERPs. When we refer to “content issue” we mean there’s also the possibility that the content of your site’s pages can affect its indexing. Some of these issues that can hinder indexing are when the content is a duplicate of some other webpage on the internet, the quality is poor, the content is too thin or it is irrelevant to users’ search intent/query.

To increase the chances of Google indexing your site’s pages ensure that your content is unique, valuable, and relevant.

Given that Google can crawl your pages, it’s a positive sign. However, indexing can sometimes take a bit longer depending on various factors, including the points above.
Here’s what you can do:

– Use the URL Inspection tool in Google Search Console to manually request indexing of your main pages.

– Double-check your content quality and ensure it’s free from issues that could affect indexing.

Please keep in mind that indexing can take time, and being patient is crucial.

We hope this helps.

Thanks.

Hello,

While you’ve mentioned there are no conflicts, sometimes certain plugins or themes can cause these kinds of issues by resetting configurations.

Try temporarily disabling all other plugins except Rank Math to see if the issue persists. If the problem goes away, re-enable the plugins one by one to identify the cause.

If the problem continues, it might be helpful to reach out to your hosting provider to ensure there are no automated scripts or settings that are causing these reverts.

Also, even though you’ve cleared the cache, there might be deeper caching mechanisms in place (like server-level caching or CDN caching) that are causing old versions of files to be served.

We hope this helps.

Thanks.

Hello,

The “no referring sitemap detected” means the page was discovered by Google just not through the sitemap.

Please note that a sitemap is only one of many ways Google discovers the URLs of your site. If Google discovers the URL through other means, it may not cross-reference your sitemap to check if the URL is there. Hence the “No referring sitemaps” message.

We shared more about this here: https://rankmath.com/kb/no-referring-sitemaps-detected/

You can also find information about the “Crawled – Currently Not Indexed” error here: https://rankmath.com/kb/crawled-currently-not-indexed/

We hope this helps.

Thanks.