Hello,
We apologize for the issue you are facing with accessing your robots.txt file. It seems like there might be a conflict or misconfiguration causing the redirection issue.
Here are a few steps you can take to troubleshoot this:
-
Check for Other Redirects: Ensure that there are no other redirect rules in your .htaccess file that might be conflicting with the robots.txt access. Sometimes, multiple rules can cause a loop.
-
Verify .htaccess Rules: Make sure that the rewrite rules you added to your .htaccess file are correct. For an Apache server, the rules should look like this:
# START of Rank Math Robots.txt Rewrite Rules
RewriteEngine On
RewriteBase /
RewriteRule ^robots.txt$ /index.php?robots=1 [L]
# END of Rank Math Robots.txt Rewrite Rules
-
Clear Cache: If you are using any caching plugins or server-level caching, clear the cache to ensure that the changes take effect.
-
Check Server Configuration: If your site is hosted on an Nginx server, ensure that the appropriate rewrite rules for Nginx are in place as well.
-
Test with a Different User-Agent: Sometimes, testing with different user-agents can help identify if the issue is specific to certain crawlers.
If you have followed these steps and are still experiencing issues, please reply to this ticket, and a human agent from the Rank Math Support team will assist you further.
Thank you for raising this concern!
Sources:
Common robots.txt Issues & How to Fix Them » Rank Math
How to Fix the ‘Blocked Due to Other 4xx Issue’ Error » Rank Math
Hello,
We checked your robots.txt file and looks like it is working properly without any issues:
Can you please confirm if you already managed to resolve the issue?
In case, you’re still facing the redirection loop, please try to clear your browser cache and check again.
Let us know how it goes. Looking forward to helping you.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
