‘noindex’ detected in ‘X-Robots-Tag’ http header

#635374
  • Resolved Release Date
    Rank Math free

    HI Team – I am trying to resolve a noindex issue for couple of days but unfortunately unable to fix it. Tried all the tips and tricks possible! Sharing below main part of the error, hoping you could help fix it.
    =======================
    Crawl allowed?
    Yes
    Page fetch
    Successful
    Indexing allowed?
    error
    No: ‘noindex’ detected in ‘X-Robots-Tag’ http header
    ===========================

    I tried taking help from WordPress & my hosting company but they both referred me to contact you as we are using your Rank Math.

    PLease help me index our post as well.

    Looking forward for your response asap. Let me know if anything else is needed from my side.

    Regards,

    Nitesh

Viewing 12 replies - 1 through 12 (of 12 total)
  • Hi Team – Any help on this is much appreciated. I have been trying to fix this for all day long, please help asap.

    Thanks

    Hello,

    Thank you for your query, and we are so sorry about the trouble this must have caused.

    First, please navigate to Rank Math > Titles & Meta > Global Meta and check if the Noindex option for your posts is active. If it is, ensure the “index” option is selected while the “Noindex” is deactivated. This will set all pages for indexing unless specified otherwise on individual pages.

    Additionally, it would be helpful to verify the Search engine visibility settings.

    Please go to WordPress > Settings > Reading and ensure “Discourage search engines from indexing this site” is not selected.

    If you continue to experience this issue, could you share the specific URLs affected? This will allow us to delve deeper and provide a more accurate solution.

    We are looking forward to helping you.

    Hi jeremy, thanks for the reply.

    I have tried both already. More than 100 times may be I checked for that and ensure its done right. But Nothing helped!

    So here is the index main file: https://releasedate.co.in/sitemap_index.xml

    The issue is only with the Post Site Map, plz see the error below
    ==============================================================
    “/post-sitemap.xml Aug 26, 2023 1 error 55 0”
    ===========================================================
    And when I click on it to check more details, I see the below error
    ========================================
    “
    Sitemap could not be read
    General HTTP error ”
    =============================

    ++ ANTHER thing I think you should know is that even though the site index link works fine when clicked or even in search console it shows ‘success’ for all pages except the ‘post site map’ – When I check on the XML Validator, it gives error and unable to validate the sitemap. SO THIS WHOLE THING IS SUPER CONFUSING!

    So I am unable to figure out what exactly is the issue with post sitemap as all the post index options are on. Really appreciate if you can make a test run on the sitemap url or do some more digging on my site related to this issue. As You guys are experts, you will be able to figure out issue!

    Waiting for your further response – Thanks!

    Hello,

    Can you please confirm if you’re inspecting your sitemap URL in your Google Search Console and getting this error? If so, then it is normal and you don’t have to worry. You’ll get the error if you inspect your sitemaps because, by default, sitemaps are set to noindex as they are meant to be crawled and not to be displayed on search results pages.

    You should only submit your sitemap to the sitemap section of your GSC account and shouldn’t use the URL inspection tool for the sitemap URLs.
    https://rankmath.com/kb/submit-sitemap-to-google/

    Hope that helps and please don’t hesitate to let us know if you have any other questions.

    Thank you.

    If that’s so then may be I should just wait and see. IN the Rank Math Youtube video also it shows that the URL can be checked if its indexed or not.

    Another issue just started from a week is my feature images are not showing when articles are shared on social media. I did follow the instructions from the Rankath guide also about Open Graph. Still no fix!

    Hello,

    Could you please share the affected URL and the image URL you wanted to use so we can check?

    Meanwhile, try clearing your website’s cache and check the og:image tag from Rank Math if it’s showing a different URL.

    Looking forward to helping you.

    Thank you.

    You can try the below link or any other from the site. When shared on twitter, feature image not coming, though its working for fb now.

    https://releasedate.co.in/webseries/american-horror-story-season-12-release-date/

    Also, would you know if this is Rank Math bug or Twitter. When you try to share on twitter, after the url in place of @ ( in via) it doesnt catch the right user name

    Thanks!

    Hello,

    We can see that the twitter:image for the page is set up correctly:

    However, most social media platforms only check the first 50Kbs of the page source for the open graph metadata. If you are using inline CSS, the og: image will not be seen by Facebook. If you have minification on and it is set to be aggressive or using inline CSS it will likely affect the tags from being seen. Kindly review your modification settings and test with this tool: https://opengraph.dev/

    On your second query you need to add space after the URL for the @ functionality on Twitter to work as it does.

    Hope this helps. Please let us know if you have further questions or concerns.

    Thanks.

    Thank you for the response.

    I disabled the minify css and WebP Image compression and also Minify Java Script Files. Still no luck!

    Could you please help figure out if this has anything to do with plugin, hosting or theme? So I can approach accordingly or try to modify things.

    It would be really helpful if you can find the root cause, I am not tech savvy. Would be grateful!

    Regards

    Hello,

    Most likely your site or server cache was presenting an older version of your site. It is a good idea to always clear the cache for new changes to reflect on the site.

    The Twitter card is now showing correctly:

    Kindly confirm this.

    Thanks.

    Release Date
    Rank Math free

    Hello Ike,

    Thanks for getting back. Yes, its all working fine now, maybe due to my change i hosting has helped.

    But I have another question related to my site map. I see 3 sitemap link in search console. Please check below.

    /post-sitemap.xml
    /sitemap_index.xml
    /sitemap.xml.
    And when I open ‘/sitemap_index.xml’ then inside this is as below:
    /author-sitemap.xml
    /category-sitemap.xml
    /page-sitemap.xml
    /post-sitemap.xml

    I must have done something wrong, but my question is – is having 3 sitemap link can cause any issue?

    If I should remove, then which one to remove and how to remove (steps)?

    Your help is greatly appreciated. I really mean it. Thanks again!

    Regards,

    Hello,

    Thank you for your patience.

    Yes, you should submit your main sitemap only, and Google will use that to crawl the inner/sub-sitemaps.

    Please remove all your submitted sitemaps and resubmit your sitemap index only:
    https://rankmath.com/kb/submit-sitemap-to-google/

    Once done, give Google some time to crawl your sitemaps.

    Hope that helps.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 12 replies - 1 through 12 (of 12 total)

The ticket ‘‘noindex’ detected in ‘X-Robots-Tag’ http header’ is closed to new replies.