google console is showing robot.txt error

#1073845
  • Resolved Nothing4u
    Rank Math free

    i have tried to fix robot.txt but it is not fixing according to google , my website is not sitemapping because of it.

    i have raisd a ticket to easy wp hosting they said to contact you and move to yoast seo

    robot.txt file
    User-agent: *
    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php
    Allow: /sitemap_index.xml
    Allow: /wp-sitemap.xml
    Sitemap: https://www.nothing4u.fun/sitemap_index.xml

    google console report

    Discovery
    Not checked in live tests
    Crawl
    Time
    25 Jul 2025, 14:06:44
    Crawled as
    Google Inspection Tool smartphone
    Crawl allowed?
    Yes
    Page fetch
    Successful
    Indexing allowed?
    error
    No: ‘noindex’ detected in ‘X-Robots-Tag’ http header
    Indexing
    User-declared canonical
    None
    Google-selected canonical

    Only determined after indexing

Viewing 3 replies - 1 through 3 (of 3 total)
  • Hello,

    I apologize for the issue you are facing with your robots.txt file and the related errors in Google Search Console. Thank you for providing the details about your current robots.txt configuration and the error message.

    From your provided robots.txt file, it seems that you have allowed access to your sitemap, which is good. However, the error message indicates that there is a noindex directive in the X-Robots-Tag HTTP header for the page you are trying to index. This means that while Google can crawl the page, it is instructed not to index it.

    To resolve this issue, you should:

    1. Check the Page’s HTTP Headers: Ensure that the page you want indexed does not have a noindex directive in its HTTP headers. You can use tools like cURL or browser developer tools to inspect the headers.
    2. Remove Noindex Directive: If you find a noindex directive, you will need to remove it from your page settings or any relevant SEO plugin settings.
    3. Verify Robots.txt: Make sure there are no conflicting rules in your robots.txt file that might prevent indexing.

    For more detailed guidance on fixing robots.txt issues, you can refer to our guide on fixing common robots.txt issues.

    If you’re still having trouble after checking these points, we recommend replying to your ticket with your website URL so that a human agent from our support team can assist you further. They can provide more specific help based on your site’s configuration.

    Thank you for raising this concern, and we hope to help you resolve it soon!

    Sources:
    Common robots.txt Issues & How to Fix Them » Rank Math
    How to Fix Indexing Issues in Google Search Console » Rank Math

    i am not able to see problem here from my side kindly help me to index sitmape to google , or do it from my side

    Hello,

    It seems that you have inspected your robots.txt, that’s why you’re getting that result.

    You can check the status of your robots.txt on the Settings > Crawling

    robots

    Also, please note that RankMath sets the “noindex” tag on your sitemaps by default. It’s done this way because a sitemap is only meant to guide search engines through your site, not to rank in search results.

    The reason you’re seeing it set to “noindex” is to prevent it from appearing in search results, which is the desired behavior.

    You should follow this guide to properly submit your sitemap:
    https://rankmath.com/kb/submit-sitemap-to-google/

    Looking forward to helping you.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 3 replies - 1 through 3 (of 3 total)

The ticket ‘google console is showing robot.txt error’ is closed to new replies.