Indexed, though blocked by robots.txt

#258062
  • Resolved Miettim

    3 years, 4 months ago

    Rank Math free

    Hello, recently found out that Google has started suddenly showing /feed pages in search results. I published blog post example http://www.site.com/howtoeat and google is also showing http://www.site.com/howtoeat/feed.

    Heard that’s not good for SEO because of duplicate content, so i disallowed feed pages with robots.txt

    My problem is now im getting Google Search Console Errors like this: Indexed, though blocked by robots.txt. How i can fix this error? Do i need to fully disable feed?

Viewing 1 replies (of 1 total)
  • Jeremy

    3 years, 4 months ago

    Hello,

    Thank you for contacting Rank Math and bringing your concern to our attention.

    Feeds are not like regular pages so you cannot noindex them through Rank Math. However, you can disallow them in your robots.txt file at WordPress Dashboard > Rank Math > General Settings > Edit Robots.txt like this which I assume you already did:

    User-Agent: *
Disallow: */feed/

    To expand my explanation, Rank Math can only configure the robots of pages that are registered in WordPress which are posts types (such as pages, posts, taxonomies, and CPTs). As for the case of the feed page, these pages are often generated dynamically when your website loads.

    At the moment, the only way to prevent this from being indexed is through the use of robots.txt.

    I hope that helps.

    Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.

    RankBot

    3 years, 3 months ago

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 1 replies (of 1 total)

The ticket ‘Indexed, though blocked by robots.txt’ is closed to new replies.