Robots.txt not fetched – redirect error

#700566
  • Resolved Neeraj Kumar
    Rank Math free

    I’m getting this issue in GSC that Robots.txt not fetched – redirect error. I believe this ks because of the code added in htaccess file to redirect non trailing slash URLs to trailing slash. But the catch is we want to have only one URL structure that is with trailing slash and without that code in htaccess its not working. Also recently there is an another issues caused by htaccess which is some of our posts on website are not getting published or updated.

Viewing 4 replies - 1 through 4 (of 4 total)
  • Hello, I’m sorry to hear about the issues you’re facing with your robots.txt file and the htaccess file. It seems like the redirect rules in your htaccess file are causing the robots.txt file to not be fetched properly.

    The issue with the posts not getting published or updated could also be related to the htaccess file.

    To resolve this, you can try removing the redirect rules from the htaccess file and then test if the robots.txt file is fetched properly. If that resolves the issue, you may need to find an alternative way to enforce the trailing slash URL structure without causing conflicts.

    If you’re unsure about how to proceed, I recommend reaching out to your web developer or hosting provider for assistance with modifying the htaccess file and resolving these issues.

    If you have any further questions or need additional assistance, please feel free to ask.

    Sources:
    Common robots.txt Issues & How to Fix Them » Rank Math
    Rank Math Sitemap Issues and Their Fixes » Rank Math
    Solving Redirection Issues Using Rank Math SEO Plugin » Rank Math

    Neeraj Kumar
    Rank Math free

    We have added the different code in htaccess file to redirect the non trailing slash urls tk trailing slash, however we have also tried to add the code from one of Rankmath articles but it works for some time and then stopped working the redirections of non trailing slash to slash urls.

    Hello,

    Thank you for your patience.

    Please take a screenshot of the error you’re getting so we can check and advise accordingly. You can upload screenshots using this tool and add the link here.

    Meanwhile, your robots.txt is already working fine without redirect so you should validate the non-trailing slash version in your GSC.

    Looking forward to helping you.

    Hello,

    We’ve checked your robots.txt file and we can see it is getting redirected from trailing slash to non-trailing slash. However, after checking some of the posts, we can see they are not getting redirected properly.

    In this case, can you please share the rule you’ve added to your htaccess file so that we can check this further for you? Meanwhile, please head over to your WordPress Dashboard > Settings > Permalinks, and hit the “Save Changes” button (without making any changes) to clear the permalink cache.

    Once done, please clear your website cache and see if that works for you.

    Let us know how it goes. Looking forward to helping you.

    Thank you.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 4 replies - 1 through 4 (of 4 total)

The ticket ‘Robots.txt not fetched – redirect error’ is closed to new replies.