Hello,
I apologize for the issue you are facing with your robots.txt file redirecting to the homepage. It seems like there might be a conflict or an additional configuration that needs to be addressed. Here are a few steps you can take to troubleshoot this issue:
-
Check for Another robots.txt File: Ensure that there isn’t another robots.txt file in your site’s root directory that could be causing the conflict. You can check this by logging into your WordPress root directory using cPanel or an FTP application. If you find another robots.txt file, delete it.
-
Clear Cache: After making changes, make sure to clear your website and server-level cache. This can often resolve issues where changes do not seem to take effect immediately.
-
Verify Rewrite Rules: Since you are using Apache, ensure that the rewrite rules you added are correct and placed properly in the .htaccess file. The rules should be above any WordPress-specific rules.
-
Test the URL: After making these changes, test your robots.txt URL again at https://yourdomain.com/robots.txt to see if it still redirects.
-
Check for CDN Issues: If you are using a CDN (like Cloudflare), it might be caching the old response. Make sure to purge the cache on your CDN as well.
If you continue to experience issues after trying these steps, please reply to this ticket with your site URL so that a human agent from our support team can assist you further. Thank you for raising this concern!
Sources:
Common robots.txt Issues & How to Fix Them » Rank Math
This is an automated reply from Rank Math Support Bot. If you need further assistance, please reply to this topic and a human support agent will reply to you shortly.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
