Google is not displaying sitemap within search results - Support

Resolved Alexander Maier
3 months, 2 weeks ago

Rank Math free

Hello Rank Math Team,

I am using the latest version of Rank Math on my website. My sitemap is generated by Rank Math at the following URL:

See under “sensitive Data”.
Unfortunately, the HTTP response header for the sitemap contains x-robots-tag: noindex. This prevents Google from crawling and indexing my sitemap properly i think.

I have already checked that no other plugins or settings are adding this header. My cache (LiteSpeed Cache) is configured not to cache the sitemap. However, the header remains.

Could you please assist me on how to remove the x-robots-tag: noindex from my sitemap or how to let google find, crawl and index my sitemap as usual?

Thank you in advance for your help!

Best regards,
Alex

Viewing 8 replies - 1 through 8 (of 8 total)

RankBot
3 months, 2 weeks ago
Hello,

We apologize for the issue you are facing with your sitemap not being indexed by Google. It seems that the x-robots-tag: noindex header is indeed preventing Google from crawling your sitemap properly.

Here are a few steps you can take to resolve this issue:
1. Check Server Configuration: Sometimes, server settings can add headers like x-robots-tag. You may want to check your server configuration files (like .htaccess for Apache or nginx.conf for Nginx) to ensure that there are no rules that are adding this header to your sitemap.
2. Contact Your Hosting Provider: If you cannot find any settings in your configuration files, it might be worth reaching out to your hosting provider. They may have security measures in place that are adding this header automatically.
3. Check Security Plugins: Although you mentioned that no other plugins are adding this header, double-check any security plugins or settings that might be affecting the headers sent with your sitemap.
4. Use a Header Checker Tool: You can use tools like httpstatus.io to check the headers being sent with your sitemap URL. This can help you confirm if the x-robots-tag: noindex is indeed present and if there are any other unexpected headers.
5. Exclude Sitemap from Caching: Since you are using LiteSpeed Cache, ensure that the sitemap is excluded from caching properly. You can refer to our tutorial on excluding sitemaps from caching for detailed instructions.
If after trying these steps the issue persists, please reply to this ticket with any additional details or findings, and a human agent from our support team will assist you further.

Thank you for raising this concern, and we hope to help you resolve it soon!

Sources:
How to Fix "Incorrect HTTP Header Content-Type" Error in Sitemaps » Rank Math
Rank Math Sitemap Issues and Their Fixes » Rank Math
Troubleshooting the “Couldn’t Fetch” Error in Google Search Console » Rank Math
Alexander Maier
3 months, 2 weeks ago

Rank Math free

Hi there,

I already contacted my hosting provider, they whitelisted some rules but the issue still persists. They told me to get in touch with RankMath regarding this issue.

When I visit: https://httpstatus.io/ the result is: “x-robots-tag: noindex”
Could you please check why the sitemap is declared as noindex?

Br, Alex
Alexander Maier
3 months, 2 weeks ago

Rank Math free
Hi,

I’ve implemented the following code into my functions.php:
```
add_filter('rank_math/sitemap/http_headers', function($headers) {
    unset($headers['X-Robots-Tag']);
    return $headers;
});
```
Seems to be working! Could you confirm, that the code is correct?
Br, Alex
Reinelle
3 months, 2 weeks ago

Hello,

Yes, that code is to set the robots of your sitemap to index.

However, please note that RankMath sets the “noindex” tag on your sitemaps by default. It’s done this way because a sitemap is only meant to guide search engines through your site, not to rank in search results.

The reason you’re seeing it set to “noindex” is to prevent it from appearing in search results, which is the desired behavior.

You should also follow this guide to properly submit your sitemap (if not yet):
https://rankmath.com/kb/submit-sitemap-to-google/

Looking forward to helping you.

Alexander Maier
3 months ago

Rank Math free

Hi Reinelle,

thank you for your quick reply!
Ahh okay I understand. But I’ve implemented the sitemap as it’s described in your guide. Unfortunately the sitemap is still not showing on google.

I don’t know why, everything seems to be set correctly.
The search engine Bing is showing the sitemap as required.

Do you have any idea?
Br, Alex

Reinelle
3 months ago

Hello,

Please share the screenshot of the sitemap on Google so we can check it.

You can upload screenshots using this tool and add the link here.

Looking forward to helping you.

Alexander Maier
3 months ago

Rank Math free

Hi Reinelle,

thank you for you reply!
As follows you can find the screenshots from google search console:
https://imgur.com/a/XrjBff4

The pages are reachable and I can click on “Apply for Indexing”. But it would be a big effort to click on each url of the affected pages.

I am looking forward to hearing from you soon
Many thanks in advance for your efforts!

Br, Alex

Ike
3 months ago

Hello,

The Crawled – currently not indexed status means Google has crawled these pages but has not indexed them. Finding a certain number of URLs with this status is entirely normal because Google does not index all the URLs we submit at the same time.

This isn’t an error but there are some reasons for excluding the pages from its index. Here’s a link for more information and how to fix it: https://rankmath.com/kb/crawled-currently-not-indexed/

There is nothing you can do to solve the Crawled – Currently not Indexed status other than wait for Google to start indexing. The crawled – currently not indexed status usually means that Google doesn’t find the content of the URLs helpful enough to be indexed in the SERPs. You can wait for Google to recrawl the URLs, or edit them and make sure that the content is optimized and helpful enough to be indexed.

Please do not hesitate to let us know if you need our assistance with anything else.

RankBot
2 months ago

Hello,

Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

Thank you.

Viewing 8 replies - 1 through 8 (of 8 total)

The ticket ‘Google is not displaying sitemap within search results’ is closed to new replies.