The sitemap generated by the plugin was not successfully crawled- noindex

#1108171
  • xinyangzr

    4 weeks, 1 day ago

    Rank Math free

    Hello, I used the plugin to generate a sitemap. The /page_sitemap.xml page in Google Webmasters shows: The system detected “noindex” in the HTTP header “X-Robots-Tag”. The generated /post_sitemap.xml can be crawled normally.

Viewing 2 replies - 1 through 2 (of 2 total)
  • RankBot

    4 weeks, 1 day ago

    Hello, we’re sorry to hear about the issue you’re facing with your sitemap. The "noindex" error in the HTTP header indicates that Google is being instructed not to index that particular page. Here are some steps you can take to resolve this issue:

    1. Check the Page Source: Open the source code of the /page_sitemap.xml and look for the X-Robots-Tag header. Ensure that it does not contain a noindex directive.

      Noindex present in page's source code

    2. Review Robots Meta Tag: Make sure that the pages you want indexed are set to index in the robots meta tag. You can check this by navigating to Rank Math SEO → Advanced tab in the post editing screen.

      index in advanced tab

    3. Sitemap Accessibility: Ensure that your sitemap is accessible. You can use tools like httpstatus.io to check if your sitemap is returning a 404 error or any other issues.

    4. Exclude Sitemap from Caching: If you have caching enabled on your site, it might be caching the sitemap incorrectly. You can refer to our tutorial on how to exclude sitemaps from caching.

    5. Check for Plugin Conflicts: Sometimes, other plugins may conflict with Rank Math. You can use the Health Check plugin to identify any potential conflicts.

    If you’ve gone through these steps and the issue persists, please reply to this ticket with your site URL so that a human agent from our support team can assist you further. Thank you for raising this concern, and we hope to help you resolve it soon!

    Sources:
    How to Fix Submitted URL Marked ‘NoIndex’ Error » Rank Math
    Rank Math Sitemap Issues and Their Fixes » Rank Math

    This is an automated reply from Rank Math Support Bot. Your ticket has been assigned to a human agent and we will get back to you shortly.

    Rakibuzzaman Siam

    4 weeks ago

    Hello,

    Please note that the sitemaps are by default set to Noindex as they are meant to be crawled and not to be displayed on search results pages. Even though it is set to Noindex, Google can crawl and read this without any issues.

    You should only submit your sitemap to the sitemap section of your GSC account and shouldn’t use the URL inspection tool for the sitemap URLs.
    https://rankmath.com/kb/submit-sitemap-to-google/

    In case you still want to index your sitemap, you can use the following filter on your website:

    add_filter( 'rank_math/sitemap/http_headers', function( $headers ) {
	if ( '/sitemap_index.xml' !== $_SERVER['REQUEST_URI'] ) {
		return $headers;
	}
	unset( $headers['X-Robots-Tag'] );
	return $headers;
} );

    Here’s how you can add a filter/hook to your WordPress site: https://rankmath.com/kb/wordpress-hooks-actions-filters/

    After applying the filter, please clear your sitemap cache by following this screencast: https://i.rankmath.com/pipRDp

    Let us know how it goes. Looking forward to helping you.

    Thank you.

Viewing 2 replies - 1 through 2 (of 2 total)

You must be logged in to reply to this ticket.