Robots.txt being recreated so Rank Math virtual robots looses effect

#958361
  • Resolved Davide

    9 months ago

    Rank Math free

    Hello, I have deleted the robots.txt file from the main directory in order to use the virtual robots from Rank Math, however it seems like another plugin or maybe WordPress itself is recreating that file and when this happens the Rank Math robots.txt is not used anymore. Am I wrong or it’s the wordpress itself that is generating the robots.txt file again? or it may be another plugin doing it?

    The robots.txt file that is created is like this:

    #Simple Robots.txt 0.1
    User-agent: MSNBot
    Crawl-delay: 5
    User-agent: bingbot
    Crawl-delay: 5

    have you noticed this behaviour in another plugin?

Viewing 2 replies - 1 through 2 (of 2 total)
  • RankBot

    9 months ago

    Hello, we apologize for the issue you are facing with your robots.txt file. It seems that another plugin or possibly a WordPress feature is indeed recreating the robots.txt file, which can interfere with the virtual robots.txt file generated by Rank Math.

    This behavior is not uncommon, as some SEO plugins or caching plugins may create their own robots.txt files. To resolve this issue, you can try the following steps:

    1. Identify the Plugin: Check if any other SEO or caching plugins are installed that might be generating the robots.txt file. You can temporarily deactivate them to see if the issue persists.

    2. Disable the Plugin: If you find a plugin that is creating the robots.txt file, consider disabling it or adjusting its settings to prevent it from doing so.

    3. Check for Theme Functions: Sometimes, themes may have functions that generate a robots.txt file. Review your theme’s functions.php file for any related code.

    4. Contact Your Hosting Provider: In rare cases, some hosting providers may have settings that automatically generate a robots.txt file. It might be worth checking with them as well.

    Once you identify and resolve the conflicting source, you should be able to use the virtual robots.txt file from Rank Math without issues. Thank you for raising this concern, and we hope this helps!

    Sources:
    I Can’t Edit My Website’s Robots.txt » Rank Math
    Common robots.txt Issues & How to Fix Them » Rank Math

    Reinelle

    9 months ago

    Hello,

    Assuming you are using the latest version of all the plugins and the themes (please update if not yet), to determine the plugin/theme that generates the robotsxt, please follow this troubleshooting guide:
    https://rankmath.com/kb/check-plugin-conflicts/

    Here is a video you can follow as well: https://www.youtube.com/watch?v=FcIWRhcUP2c

    Only the site admin would know and your users won’t be affected by this troubleshooting.

    Looking forward to helping you.

    RankBot

    8 months ago

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 2 replies - 1 through 2 (of 2 total)

The ticket ‘Robots.txt being recreated so Rank Math virtual robots looses effect’ is closed to new replies.