robots.txt issue

#959778
  • Resolved cloudworx

    7 months ago

    Rank Math free

    Hello Support,

    It has been made to our attention that there is an issue with the robots.txt file. Upon testing out site with various tools it appears the robots.txt file is placed in the public_html folder and we understand that it should be in the root.

    When checking Rank Math general settings/robots txt there is an error on line 8.

    Can you help with this?

    See attached.

Viewing 4 replies - 1 through 4 (of 4 total)
  • RankBot

    7 months ago

    Hello,

    We apologize for the issue you are facing with your robots.txt file. It seems that the presence of a physical robots.txt file in your public_html folder is causing conflicts with Rank Math’s virtual robots.txt management.

    To resolve this issue, you can follow these steps:

    1. Remove the Physical File: Access your website’s root folder using FTP or cPanel File Manager and delete the existing robots.txt file located in the public_html folder. This will allow Rank Math to manage the robots.txt file through the WordPress dashboard.

      Robots.txt file in root folder

    2. Check for Errors: After removing the physical file, go back to Rank Math SEO → General Settings → Edit robots.txt to check if the error on line 8 persists. If it does, please ensure that there are no conflicting plugins that might be creating a robots.txt file.

    3. Permissions Issue: If you still encounter issues, it might be related to permission settings on your web server. In that case, contacting your web host for assistance would be advisable.

    If you need further assistance or if the problem continues, please reply to this ticket, and a human agent from the Rank Math Support team will take over and provide more help.

    Thank you for raising this concern!

    Sources:
    I Can’t Edit My Website’s Robots.txt » Rank Math
    Common robots.txt Issues & How to Fix Them » Rank Math

    Rakibuzzaman Siam

    7 months ago

    Hello,

    Thank you for contacting us and sorry for any inconvenience that might have been caused due to that.

    We tried to check your website but were not able to access it due to the Wordfence firewall.

    Please note that Rank Math by default doesn’t add this rule to your robots.txt file. However, you can edit the robots.txt file from WP Dashboard > Rank Math > General Settings > Edit Robots.txt and remove the Host rule from there.

    If you see any error while editing the robots.txt file on your website, you can share a screenshot with us so that we can check this further for you.

    Let us know how it goes. Looking forward to helping you.

    cloudworx

    7 months ago

    Rank Math free

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    Rakibuzzaman Siam

    7 months ago

    Hello,

    We can see the Host rule is still showing on your robots.txt file.

    Since you’re using a physical robots.txt file on your website, you’ll have to directly edit this from your FTP or cPanel file manager.

    After removing the “Host” rule, please clear your website cache including any server-level cache and check again.

    Let us know how it goes. Looking forward to helping you.

    Thank you.

    RankBot

    6 months ago

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 4 replies - 1 through 4 (of 4 total)

The ticket ‘robots.txt issue’ is closed to new replies.