robots.txt not generated by rankmath

#253440
  • Resolved Ron B Palmer
    Rank Math free

    When I query my robots.txt file it returns the following:
    User-agent: *
    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php

    Sitemap: https://fixfamilycourts.com/sitemap_index.xml

    What I don’t see is any reference to RankMath generating the file. Is RankMath generating this file? How do I tell if its improperly generated somewhere else.

    Also, I am getting server errors:
    292210#0: *54540 openat() “/var/www/vhosts/fixfamilycourts.com/httpdocs/robots.txt” failed (2: No such file or directory)

    There is no physical file present. Does the openat() command require a physical file or should it trigger RankMath to generate and return the file?

Viewing 11 replies - 1 through 11 (of 11 total)
  • Hello,

    Thank you for contacting us and sorry for any inconvenience that might have been caused due to that.

    Our plugin doesn’t generate an actual physical file on your website. The file we create is dynamically created and it’s perfectly fine that you don’t see a physical file.

    ​​​​​​​Hope this helps clarify your doubts.

    Don’t hesitate to get in touch if you have any other questions.

    I didn’t have doubts about the physical file. I have a lack of knowledge of the specific mechanism RankMath uses to respond to the http request to serve the file. I am fully aware of how how non-static websites work and am fully aware that there are multiple methods available to respond to such calls.

    If you notice, the openat() command is NOT making an http request but appears to be making a direct file system request for the file specifying the full file path. So, I have two questions:
    1) when the robots.txt file resolves from an https://fixfamilycourts.com/robots.txt request, how do I know for sure that it is RankMath serving the generated file given that there are no comments displayed in the result that would indicate who generated the file? and 2) should I expect the following alternate method of requesting the file to trigger a response from RankMath: openat() “/var/www/vhosts/fixfamilycourts.com/httpdocs/robots.txt” ?

    I would like to eliminate this error and am experimenting with different ways to rewrite the request on my server or in my .htaccess file so that the error will be eliminated. It may not be a major issue because the requests are generated by robots but if google and bing bots cannot access the file then it is a significant issue. I don’t know if they try multiple methods to get the file or if they simply stop trying when this method fails.

    just for grins, presume that I have 25 or more years of high level technical experience and that I hold multiple MCSEs dating from the last century.

    Hello,

    The generated file has the correct format and is similar to the one generated by Rank Math. To make sure that it is being generated by Rank Math, please navigate to WP Dashboard > Rank Math > General Settings > Edit Robots.txt. If you are able to edit the file then the file is being generated by Rank Math.

    And the only way to access it is from https://fixfamilycourts.com/robots.txt as it is a dynamically generated file and will not respond to any other request.

    Hope this clarifies your doubts. Let us know if you need any further assistance.

    It is not writable. This is the message it displays: robots.txt file is not writable.

    Hello,

    In this case, please check if there is any static robots.txt file present in your website’s directory. If you see any, please remove it from the directory.

    Looking forward to helping you.

    from above:
    There is no physical file present. Does the openat() command require a physical file or should it trigger RankMath to generate and return the file?

    Ankit
    Rank Math business

    Hello,

    Could you please check if you have DISALLOW_FILE_EDIT is set to true ? Please set it to false. Also you can contact your host if they have defined it globally.

    Looking forward to helping you. Thank you.

    Would you kindly tell me where to actually find that setting?

    I control my own virtual server. There are no global defaults.

    Prabhat
    Rank Math agency

    Hello,

    You can find that under your wp-config.php file.

    Please also ensure that there are no permission issues that might be restricting the file from getting accessed/edited.

    Also, please verify that there’s no plugin (other than Rank Math), virtually or physically generating the robots.txt file.

    Please let us know how that goes.

    Thank you.

    DISALLOW_FILE_EDIT fixed it. Thanks.

    Hello,

    We are super happy that this resolved your issue. If you have any other questions in the future, know that we are here to help you.

    If you don’t mind me asking, could you please leave us a review (if you haven’t already) on https://wordpress.org/support/plugin/seo-by-rank-math/reviews/#new-post about your overall experience with Rank Math? We appreciate your time and patience.

    If you do have another question in the future, please feel free to create a new forum topic, and it will be our pleasure to assist you again.

    Thank you.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 11 replies - 1 through 11 (of 11 total)

The ticket ‘robots.txt not generated by rankmath’ is closed to new replies.