Hello,
We apologize for the issue you are facing with the feed pages and WebP URLs appearing in Google Search Console under "Crawled – currently not indexed."
Regarding your question, WebP images are treated differently by Google because they are image files and not HTML pages. This means that while they can be crawled, they do not get indexed in the same way as web pages do. If you see WebP URLs in the "Crawled – currently not indexed" report, it is generally safe to ignore them since they can still be discovered through Google Image Search.
To prevent these pages from being indexed or to manage how Google interacts with them, consider the following best practices:
- Noindex Meta Tag: If you want to ensure that certain pages (like those containing WebP images) are not indexed, you can add a noindex meta tag to those pages. This tells search engines not to include them in their index.
- Robots.txt: You can also use your robots.txt file to disallow crawling of specific directories or file types, including .webp files, if you do not want them to be crawled at all.
- Content Quality: Ensure that the pages you want indexed provide valuable content. If the pages are thin or lack substance, it may be better to remove them or redirect them to more relevant content.
For more detailed guidance on managing indexing issues, you can refer to our article on Crawled – Currently Not Indexed.
If you have any further questions or need assistance with specific URLs, please reply to this ticket with those details, and a human agent will assist you further.
Thank you for raising this concern!
Sources:
How to Fix "Crawled – Currently Not Indexed" Error in Google Search Console » Rank Math
How to Fix "Crawled – Currently Not Indexed" Error in Google Search Console » Rank Math
This is an automated reply from Rank Math Support Bot. Your ticket has been assigned to a human agent and we will get back to you shortly.
Yoast SEO has dedicated option to optimize crawl in a crawl optimization section where we can block feed pages irrelevant crawling of the pages. Does Rank Math have such options?
Hello,
Thank you for your query and we are so sorry about the trouble this must have caused.
You can click the feed URLs in the GSC report. You should see the referring page. This is the page where Google discovers those feed URLs.
You can either safely ignore this error on your Google Search Console or apply the following robots.txt directives:
User-agent: *
Disallow: */feed/
Here’s how you edit your robots.txt in Rank Math plugin: https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/
As for the webp images, they shouldn’t be in the Index status in Google Search Console, the indexing report is for HTML pages, not for the images.
However, even though Google has discovered your image files and crawled them, they are not supposed to be indexed in the normal search results, and that’s why they are in the Crawled – Currently not Indexed status.
In this case, you can ignore the status, as the images shouldn’t be indexed in the normal search results.
Hope that helps.
