No: ‘noindex’ detected in ‘X-Robots-Tag’ http header

#1080061
  • Hello team,
    We were using YOAST SEO but it sucks that for months our website still didn’t appear on Google even though we already submitted to Google Search Console.

    So just now we deleted that and switched to RankMath. The connection is green in the setting of RankMath plugin. May I know how do I check that if everything is connected successfully so that our website will appear on Google ASAP?

    Website: https://cepa33sg.com/
    Sitemap: https://cepa33sg.com/sitemap_index.xml

    Our email is: m********@3**************

    Thank you for your attention and support.

Viewing 6 replies - 1 through 6 (of 6 total)
  • Hello,

    We apologize for the issue you are facing with your website not appearing on Google. Thank you for switching to Rank Math and for providing your website and sitemap URLs.

    To ensure that everything is set up correctly in Rank Math and that your site can be indexed by Google, please follow these steps:

    1. Check the Rank Math Metabox: Open a post or page in the WordPress Editor. In the Gutenberg Editor, click the Rank Math SEO icon in the top-right corner to open the Metabox. For Classic Editor, scroll down to find the Rank Math Metabox. Ensure that the Robots Meta is set to Index.
      Rank Math MetaBox

    2. Verify Search Engine Visibility: Go to SettingsReading in your WordPress admin dashboard. Make sure that the Search engine visibility option is unchecked. If this option is checked, it will prevent search engines from indexing your site.
      Search engine visibility test from settings

    3. Check Your Sitemap: Ensure that your sitemap is accessible by visiting https://rankmath.com/wp-content/uploads/2022/05/Rank-Meth-Settings.png. You can also use tools like httpstatus.io to check if your sitemap is returning a 200 status code.
      Check if sitemap is accessible with the httpstatus tool

    4. Validate in Google Search Console: After making sure everything is set correctly, go to Google Search Console and check for any errors. You can also use the Validate Fix option if there were previous issues.

    5. Wait for Google to Crawl Your Site: After ensuring everything is set up correctly, it may take some time for Google to crawl your site and reflect the changes in search results.

    If you would like a more comprehensive setup or assistance, we recommend our professional setup service available at Rank Math Services. This service requires a PRO subscription and ensures that your plugin setup adheres to the latest SEO standards.

    If you have any further questions or need additional assistance, please reply to this ticket, and a human agent from our support team will assist you further.

    Sources:
    Fix Excluded by NoIndex Tag – Google Search Console Status » Rank Math
    How to Fix Submitted URL Marked ‘NoIndex’ Error » Rank Math

    Hello,

    Thank you so much for getting in touch.

    As long as you have created the property in GSC and verified your website, your website should be ready to be indexed on the SERP. Here are some guides you can follow:

    https://rankmath.com/kb/connect-your-google-account/
    https://rankmath.com/kb/google-site-verification/
    https://rankmath.com/kb/google-index/

    As for your concern about the sitemap, this is not an issue as XML Sitemaps are designed to be crawled by search engines, not indexed. The X-Robots-Tag: noindex directive ensures that the sitemap itself does not appear in search results, but this does not prevent Google from crawling its contents and indexing the pages listed within it.

    There’s no need to worry, as this is the expected behavior for sitemaps.

    Please avoid submitting your XML sitemaps to Google Search Console’s URL Inspection tool as that tool is meant for HTML pages.

    You can properly submit your sitemap to your Google Search Console account by following this guide: https://rankmath.com/kb/submit-sitemap-to-google/

    However, if you still wish to remove the noindex tag from your sitemap for any specific reason, you can use the filter in your site:

    add_filter( 'rank_math/sitemap/http_headers', function( $headers ) {
    	if ( '/sitemap_index.xml' !== $_SERVER['REQUEST_URI'] ) {
    		return $headers;
    	}
    	unset( $headers['X-Robots-Tag'] );
    	return $headers;
    } );

    Here’s how you can add filter/hook to your WordPress site:
    https://rankmath.com/kb/wordpress-hooks-actions-filters/

    Looking forward to helping you on this one.

    Hi Jeremy,

    The issues we’re having now is that even though we already verified our website on GSC and added the sitemap_index to GSC, the website pages are not appearing on Google searches at all, after months. The website is live and yet the pages are not being crawled by Google at all.

    Thus we want to check why does this happen

    Hello,

    There could be some reasons Google is not indexing the affected URLs or pages.

    Please provide a screenshot of your Google Search Console’s Page Indexing Report so we can check.

    To share the screenshot, you can use a tool like https://imgur.com/. This will allow us to better understand the situation and provide the appropriate support.

    You can also request the indexing of your pages by passing them through the URL Inspection Tool and requesting it there: https://developers.google.com/search/docs/crawling-indexing/ask-google-to-recrawl. The tool will also let you know if there are any issues with the website that prevent the pages from getting indexed.

    Meanwhile, we have provided some guides you can follow to ensure your website keeps showing up on Google search results: https://rankmath.com/kb/google-index/ and https://www.youtube.com/watch?v=_YYecHEIwMs

    Don’t hesitate to let us know if you have any other questions.

    Hi Adetayo,

    Kindly refer to this video: https://www.loom.com/share/2c27a7928ec147a49b88db01436ba713?sid=1bf096af-9ca8-421a-adee-c57e5f1b0f22

    I already tested many times using URL inspections, and the web pages are showing URL not on Google. When I use “Test Live URL”, it shows can be found, however, I already tried submitting for indexing many times as well, in the end it is still showing URL not on Google.

    The homepage cepa33sg.com even though shows URL on Google, but when I search on Google, the website cannot be found.

    Also, the sitemap_index shows 0 discovered pages.

    Hello,

    Please follow the steps below:

    1. Flush the Sitemap cache by following this video screencast:
    https://i.rankmath.com/pipRDp

    2. Exclude the Sitemap files of the Rank Math plugin in your caching plugin. The cache could be via a plugin or from the server. For plugins or Cloudflare, please follow this article:

    https://rankmath.com/kb/exclude-sitemaps-from-caching/

    3. Apply the following filter code to your site.

    
    add_filter( 'rank_math/sitemap/enable_caching', '__return_false');

    Here’s how you can add filter to your WordPress site:https://rankmath.com/kb/wordpress-hooks-actions-filters/

    4. Remove all your sitemaps from your Google Search Console and re-submit only the index sitemap (https://website.com/sitemap_index.xml)

    This will ensure that your sitemaps are up to date and Google can see the updated content as well.

    From your video screencast, the sitemaps are returning a “success” response, so it’s up to Google if they decide to crawl your website. This usually depends on your website authority.

    As for the indexing issues, please request Google to index the URLs right away by clicking on the “REQUEST INDEXING” button.

    Let us know how that goes. Looking forward to helpin you.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 6 replies - 1 through 6 (of 6 total)

The ticket ‘No: ‘noindex’ detected in ‘X-Robots-Tag’ http header’ is closed to new replies.