Hello,
Thank you for contacting Rank Math, and sorry for any inconvenience caused.
Please enable Advanced Mode in Rank Math:
Then, head over to Rank Math > General settings > Edit robots.txt and add the following rules:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://example.com/sitemap_index.xml
Replace example.com with your domain name.
Hope this helps.
Don’t hesitate to get in touch if you have any other questions.
Hi there,
Thank you for the reply. I see it was already on Advanced Mode. I have added the line you shared, but still not getting the page loading.
Any other suggestions?
Steff
Hello,
I’ve checked your robots.txt, and it is getting an HTTP error 404.
In this case, please check your Nginx config file and ensure there are no directives blocking robots.txt on your site. You can get in touch with your web host as well for assistance.
Meanwhile, the robots.txt URL I’ve shared in the sensitive data section is working to display your robots.txt. You can try adding this rewrite rule on your site and check if the /robots.txt is working properly:
# START Nginx Rewrites for Rank Math Robots.txt File
rewrite ^/robots.txt$ /index.php?robots=1 last;
rewrite ^/([^/]+?)-robots([0-9]+)?.txt$ /index.php?robots=$1&robots_n=$2 last;
# END Nginx Rewrites for Rank Math Robots.txt File
Let us know how this goes.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
