Hello,
I’m sorry to hear about the issue you’re facing with your robots.txt file after switching from Yoast SEO to Rank Math. If your robots.txt file was copied over from Yoast, it’s recommended to update it with the one created by Rank Math to ensure it aligns with your current SEO settings.
To edit your robots.txt file with Rank Math, you can follow these steps:
- Log in to your WordPress website and switch to the Advanced Mode in Rank Math’s dashboard.
- Navigate to your robots.txt file in Rank Math under WordPress Dashboard → Rank Math SEO → General Settings → Edit robots.txt.
- Add or edit the code in the available text area as needed.
- Save your changes by clicking on the Save Changes button.
If you’re unable to edit the robots.txt file using Rank Math, please ensure that you are in the Advanced Mode. If the issue persists, you can assign this ticket to a human agent by clicking on the appropriate button so that they can assist you further.
Thank you for bringing this to our attention, and we appreciate your patience as we work to resolve this issue for you.
Sources:
How to Edit Your Robots.txt With Rank Math SEO » Rank Math
How to Fix ‘Blocked by robots.txt’ Error in Google Search Console » Rank Math
The #1 WordPress SEO Plugin in 2023 – Rank Math
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
I was not able to find the advanced mode
Hello,
Thank you for contacting us and sorry for any inconvenience that this issue might have caused.
The robots.txt on your site is not generated by Rank Math.
Rank Math generates the robots.txt for your site dynamically. You need to delete any physical robots.txt file in your root domain for Rank Math to generate it dynamically.
Let us know how this goes. Also, let us know if you have further questions or concerns.
Thank you.
How can I delete and have rank math create a new one?
Ok done was able to delete my physical robots.txt file in your root domain do i need to do anything else for rank math to make a new one?
Hello,
Thank you for your patience.
We’ve checked your robots.txt, and it still contains rules that Rank Math does not add by default.
Here are the only rules we add to our default and recommended robot.txt:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: yoursite.com/sitemap_index.xml
Please ensure that there is no robots.txt file existing in your root directory. Your website’s root directory contains folders like wp-admin, wp-content, etc.
Once done, Rank Math will create a default robots.txt with the rules we shared above.
Let us know how this goes.
Hello i alrewdy deleted my robots.txt on my root directory. Upon checking i can now edit the one on rank math do i manually delete it or enter the one on your screenshot?
Hello,
You should delete the text inside, put the code we’ve shared there, and save the changes.
Once done, open your robots.txt URL and check if it’s getting applied correctly.
Please clear your website and browser’s cache as well to see the changes.
Let us know how this goes.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
