Indexed, not submitted in sitemap

#247474
  • Resolved Swogas
    Rank Math free

    Hey,
    I am facing a problem that from 08-08-2021 my sitemaps are not being updated and my posts are also not being indexed and I am losing the traffic because of that.
    before I was using Yoast SEO plugin and I was not facing issues that much. but then I switched to Rankmath and I was hoping that it will be the best option for me.
    but I am facing a serious issue regarding sitemaps, as I did every possible effort to solve this issue but it’s not helping me out.

    I am facing several issues such as:

    Crawled – currently not indexed
    Excluded by ‘noindex’ tag
    Discovered – currently not indexed
    Duplicate without user-selected canonical
    Alternate page with proper canonical tag

    Even I tries robots.txt Tester too but this result was shown up:

    You have a robots.txt file that we are currently unable to fetch. In such cases we stop crawling your site until we get hold of a robots.txt, or fall back to the last known good robots.txt file

    Kindly help me out as I am losing my traffic
    Thank You

Viewing 15 replies - 1 through 15 (of 18 total)
  • Prabhat
    Rank Math agency

    Hello,

    Thank you for contacting us and we regret the inconvenience caused.

    Could you please share the URL of your website so that we can check this further?

    Looking forward to helping you.

    Thank you.

    Swogas
    Rank Math free

    Sure,
    https://www.swogas.com/

    Kindly please check all the issues and guide me through their solutions. It would be much appreciated.
    Thank You

    Anas
    Rank Math business

    Hello,

    Indexed, Not Submitted in Sitemap is not an issue, it only means that Google has already indexed the posts/pages but not through the sitemap.

    It is possible that Google is crawling your website and indexing the posts, pages through the internal links on your website and not through the sitemap, and as a result, the crawl budget for your website is exhausted and Google is not able to discover the URLs through your sitemap.

    There is nothing to worry about as long as the posts/pages are getting crawled and indexed.

    Please exclude the Sitemap files of the Rank Math plugin in your caching plugin. The cache could be via a plugin or from the server. For plugins or Cloudflare, please follow this article:
    https://rankmath.com/kb/exclude-sitemaps-from-caching/

    Once done, clear the website cache and resubmit sitemap_index.xml and wait for Google to crawl the sitemap.

    I checked your robots.txt file, please edit the robots.txt file and enter the sitemap in a new line. Your robots.txt file should look like this:

    User-agent: *
    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php
      
    Sitemap: https://www.swogas.com/sitemap_index.xml

    Here is a guide for your reference:
    https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/

    I hope this helps. Looking forward to helping you.

    Swogas
    Rank Math free

    hey.
    thankyou for your support, I want to tell you that I have followed the stuff you said

    1. I entered to mentioned code in robot.txt
    2. I edit the htaccess with your code given
    3. then I delete the previous sitemap from the search console and submit a new one

    these things I have done are it sufficient? if any are missing kindly tell me as I don’t want anything to be left.

    Secondly, you told me to exclude the sitemap from the cache plugin but I don’t use any plugin mentioned in your article, I use SG Optimizer which is from my hosting Siteground Default and I don’t know which Url to put there. Can you guide me from it?

    And also I skip this step because in your article it was mentioned that editing access will help most that’s why I didn’t do the rest of the steps.

    this is all I did please guide me if I did anything wrong or I miss anything?

    Thank You So Much

    Hello,

    Yes, those steps you have done are sufficient for Google to recrawl your sitemap. You’ll just need to give Google some time to recrawl your whole site using the resubmitted sitemap.

    If you’re not using a caching plugin on your site, you can get in touch with your web host to assist you in excluding your sitemaps from being cached on a server level.

    I hope that helps.

    Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.

    Swogas
    Rank Math free

    Can you please provide me the Proper Url of the sitemaps of rankmath that I can provide to my hosting support so they can exclude those urls.
    Thankyou

    Prabhat
    Rank Math agency

    Hello,

    The URL of the index (main) sitemap on your website is https://www.swogas.com/sitemap_index.xml.

    You should ask your host to exclude all URLs, containing the word sitemap, from getting cached. This would exclude all the sitemaps including the sub-sitemaps.

    Hope this helps.

    Thank you.

    Swogas
    Rank Math free

    Okay Thank You So much for your help

    Swogas
    Rank Math free

    But should I do it now? Or I have to wait that after doing all the rest settings I mentioned above except excluding urls, might google start to crawl my posts?

    Prabhat
    Rank Math agency

    Hello,

    Excluding the URLs from getting cached should be done along with all the other steps.

    Since the sitemap should not remain cached when search engine bots crawl your website, you can contact your host and get the sitemaps excluded.

    Hope this helps.

    Thank you.

    Swogas
    Rank Math free

    I have excluded this URL https://www.swogas.com/sitemap_index.xml from the cache. As my hosting support team was not dealing in SEO service so I didn’t get much help.
    kindly tell me is this enough to exclude this URL because I saw in your article that there were many codes and URLs you edit in different plugins so kindly tell me if something else is required and some more URLs need to be excluded.
    and also confirm me that after excluding this URL does it will affect my website in any way?
    thank You

    Anas
    Rank Math business

    Hello,

    I checked the sitemaps, Cloudflare is caching the sitemaps.

    Please exclude the sitemaps from caching in Cloudflare by following this guide:
    https://rankmath.com/kb/exclude-sitemaps-from-caching/#cloudflare

    Once done, purge the Cloudflare and resubmit the sitemap_index.xml again.

    Excluding the sitemaps from caching will not affect the website negatively.

    I hope this helps. Looking forward to helping you.

    Swogas
    Rank Math free

    I have removed the Cloudflare form my SG Optimizer as it was not needed. Kindly now check it everything alright as i have submitted the sitemap to search console agian.
    If there is any other issue kindly tell.
    Thank You

    Hello,

    The sitemap seems to be working fine now.

    Please head over to your Google Search Console, remove all the sitemap/s added there, and re-submitted the primary sitemap which is https://www.swogas.com/sitemap_index.xml.

    Give it some time for Google to recrawl your website and see if the issue goes away.

    https://rankmath.com/kb/submit-sitemap-to-google/

    Let us know how it goes. We are looking forward to helping you.

    Swogas
    Rank Math free

    Yes already done Thankyou so much for your great support.

Viewing 15 replies - 1 through 15 (of 18 total)

The ticket ‘Indexed, not submitted in sitemap’ is closed to new replies.