Hello,
We apologize for the issue you are facing with your robots.txt file after migrating your website. It seems that the old sitemap URLs are still present in your robots.txt file, which can cause confusion for search engines.
To resolve this, you will need to update your robots.txt file to reflect the new URLs. Here are the steps you can follow:
-
Access the robots.txt file: Since you mentioned that the contents are locked because a robots.txt file is present in the root folder, you will need to either edit this file directly via FTP or your hosting control panel, or you can use the Rank Math plugin if it allows you to manage it from the dashboard.
-
Update the Sitemap URLs: Change the old sitemap URLs to the new ones:
-
Save Changes: After making these changes, save the robots.txt file.
-
Verify Changes: You can use the Robots Testing Tool to ensure that your new robots.txt file is correctly configured and accessible.
If you encounter any issues while editing the robots.txt file or if it remains locked, please reply to this ticket, and a human agent from the Rank Math Support team will assist you further.
Thank you for raising this concern!
Sources:
Common robots.txt Issues & How to Fix Them » Rank Math
I have updated the url’s of the robots.txt file via FTP. Do we need to take any further actions?
User-Agent: *
Allow: /wp-content/uploads/
Disallow: /wp-admin/
Disallow: /readme.html
Disallow: /refer/
Sitemap: https://autel.us/sitemap_index.xml
Sitemap: https://autel.us/page-sitemap.xml
Hello,
You should only include your sitemap_index.xml in your robots.txt rules as we have recommended in our default rules here:
https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/#default-rules
Hope that helps.
Hello,
We are super happy that this resolved your issue. If you have any other questions in the future, know that we are here to help you.
If you don’t mind me asking, could you please leave us a review (if you haven’t already) on https://wordpress.org/support/plugin/seo-by-rank-math/reviews/#new-post about your overall experience with Rank Math? We appreciate your time and patience.
If you do have another question in the future, please feel free to create a new forum topic, and it will be our pleasure to assist you again.
Thank you.
