No: ‘noindex’ detected in ‘X-Robots-Tag’ http header

#215740
  • Discovery
    Not checked in live tests
    Crawl
    Time
    Jun 20, 2021, 4:07:57 PM
    Crawled as
    Googlebot smartphone
    Crawl allowed?
    Yes
    Page fetch
    Successful
    Indexing allowed?
    error
    No: ‘noindex’ detected in ‘X-Robots-Tag’ http header
    Indexing
    User-declared canonical
    None
    Google-selected canonical
    
    Only determined after indexing

Viewing 15 replies - 1 through 15 (of 16 total)
  • Hello,

    Thank you for contacting Rank Math and bringing your concern to our attention. I’m sorry for any inconvenience this issue may have caused you.

    Could you please confirm if you’re checking your sitemap URL? If so, sitemap URLs (/sitemap_inde.xml) are set to noindex by default since you don’t want them showing on the search results pages.

    Search engines use them to crawl the URLs included in your sitemap.

    If not, please also share the affected URL to check further.

    I hope that helps.

    Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.

    I attach the picture of my sitemap have a look! and please give a solution.

    Anas
    Rank Math business

    Hello,

    You seem to be using Yoast for your sitemaps. (Check screenshot in sensitive data section)

    Please disable Yoast and enable the Sitemap module in the Rank Math dashboard.
    sitemap

    Then, follow these steps:
    1. Flush the Sitemap cache by following this video screencast:
    https://i.rankmath.com/pipRDp

    2. Exclude the Sitemap files of the Rank Math plugin in your caching plugin. The cache could be via a plugin or from the server. For plugins or Cloudflare, please follow this article:
    https://rankmath.com/kb/exclude-sitemaps-from-caching/

    Once done, remove the sitemaps from Search Console and add sitemap_index.xml and wait for Google to crawl the sitemap.

    The couldn’t fetch error, is in most cases, simply due to a bug in the Google Search Console.

    Please follow this guide to fix the issue:
    https://rankmath.com/kb/couldnt-fetch-error-google-search-console/

    I hope this helps. Looking forward to helping you.

    I have done all steps you have been recommended. Let’s watch it’s working or not.

    Ankit
    Rank Math business

    Hello,

    Thank you for your reply.

    You may need to flush your sitemap with the w3 cache plugin because I still see the old sitemap, and also, as my colleague mentioned, Exclude the Sitemap files of the Rank Math plugin in your caching plugin.

    See the screenshot for your reference:
    https://i.rankmath.com/d864qV

    Looking forward to helping you. Thank you.

    Hey there, Here is my update.
    my post sitemap is successfully indexed, but my page sitemap is still showing “Sitemap could not be read.”

    i already do it before last sitemap resubmit,

    Here is my update. I completely delete the Yoest SEO plugins file from Cpanel and PHPMyAdmin database. but it still showing the Yoast SEO generate sitemap 😥😥😥
    What should i do?

    Anas
    Rank Math business

    Hello,

    We might need to take a closer look at the settings. Please edit the first post on this ticket and include your WordPress & FTP logins in the designated Sensitive Data section.

    Please do take a complete backup of your website before sharing the information with us.
    Sensitive Data Section

    It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:

    https://wordpress.org/plugins/temporary-login-without-password/

    You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):

    https://wordpress.org/plugins/wp-security-audit-log/

    We really look forward to helping you.

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    Hello,

    Thank you for that information.

    I’ve checked your sitemap using httpstatus.io, and it is returning to 403. I’ve attached a screenshot in the sensitive data section. In this case, you can get in touch with your web host to check this issue for you. Your sitemap should show 301 for the 3 versions and 200 OK status.

    Also, please check and remove any static sitemap file in your site’s root directory as Rank Math’s sitemap is generated virtually.

    Once done, remove all your submitted sitemaps and resubmit your main sitemap only /sitemap_index.xml and give Google some time to recrawl your site.

    I hope that helps.

    Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.

    Can I use the 301 Redirects – Easy Redirect Manager plugin? The XML file does not redirect to 301. I can redirect to 301 by this plugin. Which links had to be redirected?

    Ankit
    Rank Math business

    Hello,

    Thank you for your reply.

    After flushing the cache from your website, I am able to see the Rank Math sitemap, and redirection is also working.

    See the screencast:
    https://i.rankmath.com/txleXc

    https://i.rankmath.com/dvYCxE

    Could you confirm the same on your end?

    Looking forward to helping you. Thank you.

    i still see the Yoast SEO generated sitemap

    I checked from incognito mode. Still showing the Yoast SEO sitemap.

    And I can’t solve the 401 redirect problem. What solution for that?

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 15 replies - 1 through 15 (of 16 total)

The ticket ‘No: ‘noindex’ detected in ‘X-Robots-Tag’ http header’ is closed to new replies.