Coverage issues – robots.txt

#265416
Viewing 5 replies - 1 through 5 (of 5 total)
  • Hello,

    Thank you for contacting the support, and sorry for any inconvenience that might have been caused due to that.

    I believe there’s an issue with your robots.txt files. You are allowing all bots to crawl your website and disallowing your pages at the same time. Please refer to my screenshot here: https://i.rankmath.com/DzgTcO

    Please head over to your WordPress Dashboard > Rank Math > General Settings > Edit robots.txt and in here, edit your robots.txt file and ensure you have the recommended configuration:

    User-Agent: *
    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php
    
    Sitemap: https://bdcpropertypartners.com.au/sitemap_index.xml

    Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.

    Jeff
    Rank Math free

    Hi Jeremy,

    Thanks for your prompt reply. I had already tried this but I can’t edit it:

    robots.txt file is not writable.

    Contents are locked because a robots.txt file is present in the root folder.

    Is there another way I can rectify this? Should I edit the file via Siteground File Manager?

    Thanks,
    Kind Regards,
    Jeff

    Hello,

    Yes, it is likely that there may be a physical robots.txt file present in your website’s directory. Please delete it so Rank Math can take over your website’s robots.txt.

    Looking forward to helping you.

    Jeff
    Rank Math free

    Hi Jeremy,

    Thanks for your reply. I deleted robots.txt and Rank Math created the correct one. I am still not seeing any data in the Rank Math dashboard. It was last updated Sept 25 2021.

    I also still can’t submit a sitemap in Google Search Console/Sitemaps, there is an error “Couldn’t Fetch” for both https://www.bdcpropertypartners.com.au and https://bdcpropertypartners.com.au.

    Can you please assist me to rectify this or suggest who I need to contact for support?

    Thanks very much for all your assistance.

    Hello,

    Can you please follow these steps?

    1. Flush the Sitemap cache by following this video screencast:
    https://i.rankmath.com/pipRDp

    2. Exclude the Sitemap files of the Rank Math plugin in your caching plugin. The cache could be via a plugin or from the server. For plugins or Cloudflare, please follow this article:
    https://rankmath.com/kb/exclude-sitemaps-from-caching/

    3. Remove all your sitemap/s from Google Search Console and submit only the primary sitemap (sitemap_index.xml).

    Let us know how it goes. We are looking forward to helping you with this one.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 5 replies - 1 through 5 (of 5 total)

The ticket ‘Coverage issues – robots.txt’ is closed to new replies.