Hello,
I apologize for the inconvenience you are facing with the ‘noindex’ error on your website. To fix this issue and ensure that only URLs containing ‘feed’ are not indexed while all other URLs are indexed normally, you can follow these steps:
-
Check Rank Math’s Titles & Meta Settings:
- Navigate to WordPress Dashboard → Rank Math SEO → Titles & Meta.
- Ensure that the settings are as per your preference, especially for the ‘noindex’ tag.
- Confirm that only URLs containing ‘feed’ are set to ‘noindex’, while all other URLs are set to be indexed.
-
Removing NoIndex with Bulk Actions:
- If multiple posts are incorrectly set to ‘noindex’, you can use Rank Math’s bulk actions feature.
- Head over to the WordPress Posts menu and filter by Articles Noindexed using Rank Math.
- Select the posts you want to set to ‘index’ and choose Set to Index from the bulk actions drop-down.
By following these steps, you can ensure that only URLs containing ‘feed’ are not indexed, while all other posts on your website are indexed normally.
If you need further assistance or have any other questions, please feel free to ask.
Thank you.
Sources:
How to Fix Submitted URL Marked ‘NoIndex’ Error » Rank Math
How to Fix ‘Page Indexed Without Content’ Issue in Google Search Console » Rank Math
How to Fix the "Indexed, though blocked by robots.txt" Error » Rank Math
Hello,
Thank you for contacting us and sorry for any inconvenience that might have been caused due to that.
We checked the page you’ve shared and we can see the robots.txt meta is already set to index. Please check the screenshot in the sensitive data section for reference.
In this case, please try to clear your website cache including any server-level cache, and revalidate the issue. Once done, give Google some time to recrawl your site.
Let us know how it goes. Looking forward to helping you.
Thank you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hi
I noticed that the Format Archives Robots Meta, Author Robots Meta, misc pages, and Tag Archives Robots Meta settings are set to noindex.
I have selected the indexing option for the Format Archives Robots Meta and Author Robots Meta. Beside, I also select the posts I want to set to ‘index’ and choose Set to Index from the bulk actions drop-down. After following these steps, will my posts be indexed in the future again? I have sent screenshots in the sensitive data section.
Regarding misc pages and Tag Archives Robots Meta, I am unsure whether to choose to index these sections or not. Please provide your suggestions.
Thank you.
Hi,
When I click on Titles and Meta, there are many options, and I’m not sure which ones to set to index or noindex. Can you suggest what I should do? I want to index all posts on my website and set noindex for URLs that contain /feed/.
Thank you
Hello,
These new changes will be effected when Google recrawl your website.
You can request the indexing of your pages by passing it through the URL Inspection Tool and request it there: https://developers.google.com/search/docs/crawling-indexing/ask-google-to-recrawl. The tool will also let you know if there are any issues on the URL that prevent them from getting indexed.
Regarding the options in the Misc. page settings:
Noindex Search Results: we recommend that you turn this setting on and No index your search results pages.
Noindex Subpages: Similar to posts and pages, archive pages can be paginated too. With this option, you can either enable or disable the no-indexing of archive subpages. Enabling this option will Noindex all the subpages, and they will not show up in the search results.
For tag archives: it is generally recommended to set the Tag Archives Robots Meta to Index. This setting allows search engines to crawl and index these pages, potentially improving their visibility in search results.
We hope this helps. Feel free to ask if you have any further questions or need assistance.
Thank you.
Hello,
In addition to your new query,
Please follow this guide in the Knowledge Base where we shared insights into all the options available in Title & Meta settings: https://rankmath.com/kb/titles-and-meta/
When you set up your Titles and Meta settings, it creates a default set of settings that will be used for posts, pages, and other post types that you might have. This ensures that each post on your website has some optimization done for them.
Please add the filter below to your website to set the feed URLs to noindex:
add_action( 'template_redirect', 'noindex_feeds' );
function noindex_feeds() {
if ( ! is_feed() ) {
return;
}
header( 'X-Robots-Tag: noindex', true );
You can follow this guide to add the filter: https://rankmath.com/kb/wordpress-hooks-actions-filters/
Alternatively, you can disallow bots from crawling the feed URLs from the robots.txt file at WordPress Dashboard > Rank Math > General Settings > Edit Robots.txt
User-Agent: *
Disallow: */feed*
Once done, give Google some time to recrawl your site to reflect the changes.
Hope that helps and please don’t hesitate to let us know if you have any other questions.
Thank you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hi
When I used Link Redirect Trace to check, I found that all my sites might be set to noindex. Could you show me where I can turn off the noindex setting?
I also followed your instructions to disallow the feed in robots.txt and pasted the code into the theme. However, when I submit the post link back to Google Search Console, it still reports an error: “No: ‘noindex’ detected in ‘X-Robots-Tag’ HTTP header.” I don’t know what to do next. Can you support me?
I have attached the screenshots in the sensitive data.
Hello,
We’ve checked your posts and pages, and they are already set to index. Your sitemaps and robots.txt are also working fine.
Please note that Google assigns a crawl budget to your website depending on these factors (especially these two) and that has a direct effect on how soon or how late your content can get indexed. It depends on a lot of factors. Your posting frequency + the domain authority are just two of the many factors Google considers when indexing some new URLs.
If Google still doesn’t decide to index those pages even after crawling your site again, then there is nothing you can do other than wait.
Here’s a link for more information:
https://www.searchenginejournal.com/definitive-list-reasons-google-isnt-indexing-site/118245/
Since your website is already set to index, then it is just a waiting game before Google decides to include the URL in the search results pages. No SEO plugin can force Google to index something they don’t want to.
With that said, we have found that barring any issues like copied/DMCA/low-quality content issues. You may check this section in your GSC for that:
https://support.google.com/webmasters/answer/9044175?hl=en
If there are no manual actions imposed on your site, we request you kindly wait for the SERP to include your site on search results pages.
Looking forward to helping you.
Hi,
My site used to have many posts and pages indexed, but recently, when I checked again, none of them were indexed. When I checked on Google Search Console, it reported ‘noindex’ detected in the ‘X-Robots-Tag’ HTTP header.
I suspect the problem is due to the ‘noindex’ tag in the ‘X-Robots-Tag’ HTTP header, but I don’t know how to change this tag from ‘noindex’ to ‘index’.
Besides the Titles and Meta section, is there anywhere else I can set the index and noindex for posts or pages in WordPress?
Hello,
We checked again and can confirm that your website is already set to index. There are mainly two ways to set your feed to noindex and none of them will affect other posts.
You can set your feed to noindex using this code:
add_action('template_redirect', 'noindex_feeds');
function noindex_feeds() {
if (is_feed()) {
header('X-Robots-Tag: noindex', true);
}
}
Or by adding this DISALLOW rule to your robots.txt file:
User-Agent: *
Disallow: */feed*
Both of these options will not affect other pages, only feeds. And the DISSALOW rule is already present on your robots.txt file.
If you are still getting noindex on a page, please share the direct URL to the affected page with us and we’ll be happy to help investigate it.
Besides the Titles and Meta section, is there anywhere else I can set the index and noindex for posts or pages in WordPress?
Go through each post type and set to index the page type that you want Google to index, i.e. your posts, pages, landing pages, portfolio, and any other relevant post type.
Thank you for choosing Rank Math
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hi
When I clicked on RankMath => SEO Analyzer, I found two notifications:
_ In the Blog Public section, it reported that search engines can’t see my website and suggested I turn off the “Discourage search engines from indexing this site” option. However, I have already turned off this option a long time ago.
_ In the Noindex Meta section, it reported that my homepage contains a noindex meta tag or header.
Therefore, I guess the noindex tag might be set somewhere else on my website. I have attached the screenshot of the SEO Analyzer above.
Hello,
When I turned off the Archives Robots Meta Tag, most of the website URLs I submitted to Google Search Console were accepted. However, the URL below was refused due to detecting “noindex” in the ‘X-Robots-Tag’ HTTP header.
Please help me check this link:
https://fastcredit.vn/kien-thuc-rut-tien/the-tin-dung-vietinbank/
I have attached the screenshot of google search console
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
