My New URL is not getting Indexed in Google Search cosole

Viewing 3 replies - 1 through 3 (of 3 total)
  • Hello,

    Thank you for contacting Rank Math today.

    I have checked it and the page seems fine, it has no no-index set so it should index without issues.

    But I found some issues in your robots.txt file. You don’t need the: Allow: /sitemap.xml since Rank Math doesn’t add that URL (it uses sitemap_index.xml) and in fact you don’t want (never) your XML sitemaps indexed, so in any case you wouldn’t set it as Allow.

    But make sure to include these lanes in your robots.txt file (changing that domain name with yours):

    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php
    Sitemap: http://mywebsite.com/sitemap_index.xml

    Looking forward to helping you. Thank you.

    ​​​​​​

    Thanks that URL error is solved.

    But as you said that sitemap shouldn’t be indexed then why we allowed that sitemap_index.xml?

    And, What’s that Admin-ajax.php?

    Hello,

    We used the Sitemap: rule, not the Allow: one. It is used to let search engines know where is the sitemap (if they don’t have it using other techniques like adding it in Search Console).

    This file is often used front-end by themes and plugins. It was changed in WordPress robots.txt file a couple of years ago to allow crawling for this reason, after an issue with google being unable to crawl it.

    The WP Core ticket thread can be found here and offers further information https://core.trac.wordpress.org/ticket/33156

    Looking forward to help you.

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this ticket.