robots.txt – not generating?

#359960
  • Resolved Matt
    Rank Math free

    Greetings,
    Setting up RankMath with wordpress on a nginx server.
    I have no issue with my sitemap being generated by rank maths.
    But I can’t seem to find my robotos.txt url. I see in the sitemaps settings it’s supposed to be virtual. I don’t understand how this is supposed to work. But there doesn’t seem to be a url with the robotos.txt being generated?
    My website is https://store.mrlb.com.au
    Could I get some advice to troubleshoot?
    Thanks, Matt

Viewing 9 replies - 16 through 24 (of 24 total)
  • Hello,

    Sorry for the late reply as we were debugging the situation further.

    The FTP access is still not available and we can’t log in using those credentials.

    Having said that, we also noticed that you don’t seem to have a default WordPress installation. The wp-config.php file and wp-content folder seem to leave elsewhere on the server and are symlinked to the root directory.

    Could you please clarify what type of installation you are using so that we can replicate it from our end and to the testing for this feature locally?

    Looking forward to hearing back from you.

    Matt
    Rank Math free

    Hey Miguel

    I am using a stock image built by bitnami. You can get a deployable image here to lab > https://bitnami.com/stack/wordpress-pro
    There are using symlinks for some reason for the wp-config.php file and wp-content for some reason.

    Strange about the ftp. I just double checked the details i provided in sensitive area and I get an ftp connection in filezilla. Please see screenshot in sensitive area.

    Cheers, Matt

    Hello,

    Looking at the configuration of your server it seems that the missing physical file is being intercepted by Nginx and you should make sure it’s caught by PHP instead so that our plugin can generate it correctly.

    We recommend consulting with your systems administrator before doing any of this, however, this sample code should be able to achieve this:

    
    location = /robots.txt {
        try_files $uri $uri/ /index.php?$args;
        access_log off;
        log_not_found off;
    }
    

    Hope this helps clarify your doubts.

    Don’t hesitate to get in touch if you have any other questions.

    Matt
    Rank Math free

    Woohoo! I can happily say this is now resolved thank you!

    I already had a “location = /robots.txt {” but was missing “try_files $uri $uri/ /index.php?$args;”

    I now understand how the virtual element to how this file is being generated now.

    I appreciate all the support efforts of rank math team.

    I noticed that there is a mention of nginx config required in the ‘sitemap’ settings of the plugin. I suspect this robots.txt issue could catch other nginx users out again in the future. Is it worth creating a KB article or make mention of the required config in the plugin?

    Hello,

    I am glad that we could address your concern.

    As for your additional input, we have taken note of it.

    We are constantly improving our KB article and let’s hope that your input will be documented for future users that will encounter this type of issue.

    With that said, thank you, and please do not hesitate to let us know if you need our assistance with anything else.

    Matt
    Rank Math free

    Your team and support has been great. I will resolve the ticket.
    But before I do is there an official way to make a feature request/improvement?

    Hello,

    Usually, our users simply submit their suggestions here in our support forum. If you have further suggestions, you can just create a new ticket here. We would be glad to accept it.

    Thank you.

    Matt
    Rank Math free

    Will do thanks!

    Hello,

    We are super happy that this resolved your issue. If you have any other questions in the future, know that we are here to help you.

    If you don’t mind me asking, could you please leave us a review (if you haven’t already) on https://wordpress.org/support/plugin/seo-by-rank-math/reviews/#new-post about your overall experience with Rank Math? We appreciate your time and patience.

    If you do have another question in the future, please feel free to create a new forum topic, and it will be our pleasure to assist you again.

    Thank you.

Viewing 9 replies - 16 through 24 (of 24 total)

The ticket ‘robots.txt – not generating?’ is closed to new replies.