Hello,
I’m sorry to hear about the issue you’re facing with managing feeds on Search Console. To address the problem of feeds appearing under “Page crawled but not currently indexed,” you may need to ensure that your robots.txt file allows these pages to be indexed.
To create a rule in the robots.txt file using Rank Math, please follow these steps:
-
Log in to your WordPress website and switch to the Advanced Mode from Rank Math’s dashboard.
-
Navigate to your robots.txt file in Rank Math located under WordPress Dashboard → Rank Math SEO → General Settings → Edit robots.txt as shown below:
-
In the robots.txt file, you can add rules to allow specific bots like Googlebot to crawl your site. You can use the following entry to allow access for the Google News crawler:
User-agent: Googlebot-news
Allow: /
By adding this rule, you explicitly allow Googlebot-news to crawl your site, which can help in getting your feeds indexed properly.
If you need further assistance or have any more questions, please feel free to ask.
Thank you for reaching out and raising this concern.
Sources:
How to Get Your Website Listed in Google News » Rank Math
Discovered – Currently Not Indexed in Google Search Console » Rank Math
How to Fix the “Indexed, though blocked by robots.txt” Error » Rank Math
How to Fix ‘Blocked by robots.txt’ Error in Google Search Console » Rank Math
Why Google Hasn’t Indexed Your Post, Page, and/or Website? » Rank Math
Hello,
Thank you for your query and we are so sorry about the trouble this must have caused.
The feed URLs are something Google shouldn’t crawl but somehow, it got discovered for indexing, hence you are receiving such report.
You can disallow bots from crawling the feed URLs from the robots.txt file at WordPress Dashboard > Rank Math > General Settings > Edit Robots.txt
User-Agent: *
Disallow: */feed
Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.
I used this operator from Google. I could also try with inurl. It appears to have no indexing issues. But I’m afraid that URLs with /feed/ could consume search engine budgets.
https://imgur.com/a/QU6uupD
Hello,
In that case, Google may still discover those feed URLs somewhere on your website or even outside your website.
To determine that, try submitting one of the feed URLs in your GSC’s URL inspection tool and share a full screenshot of the report here.
Looking forward to helping you.
Hello,
The referral page for your feed is no longer active and it seems that Google crawled this URL last year in December, meaning, you just need to let Google crawl this again and update the GSC report, this error should go away until then.
Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.
“meaning, you just need to let Google crawl this again and update the GSC report, this error should go away until then.”
I would like to know how to do this, since I have had problems for 3 years and on 2 sites…
Hello,
You can also request the indexing of your pages by passing it through the URL Inspection Tool and request it there: https://developers.google.com/search/docs/crawling-indexing/ask-google-to-recrawl.
Hope this helps and please don’t hesitate to let us know if you have any other questions.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
