Hello,
Thank you for contacting Rank Math and sorry for any inconvenience that might have been caused due to that.
It is likely you may be having a robots.txt file existing on your site in which case Rank math doesn’t generate the virtual version. To confirm this please head over to Rank math > General Settings > Robots.txt and check it is editable from here. If it is not, then you do have a robots.txt file present in your root folder. You can delete or rename this to allow Rank math to generate one dynamically.
As for the issue you are facing with 404 monitor, assuming you are using the latest version of all the plugins and the themes (please update if not yet), it would seem like a conflict with one of the plugins or the theme you are using.
To determine that, please follow this troubleshooting guide:
https://rankmath.com/kb/check-plugin-conflicts/
Please let us know if you find the culprit.
Only the site admin would know and your users won’t be affected by this troubleshooting.
Looking forward to helping you.
Hi Micheal, thank you for your reply. I did some searching before posting this support ticket, so i’ve checked the basics. Maybe i need to rephrase the question.
Robots.txt is available when i directly access it. The visitors logs on Cpanel SHOW a 404 error. Rank Maths’s 404 monitor DOES NOT.
I do not know if this is normal for a virtual robots.txt.
Hello,
Thank you for the follow up.
We might need to take a closer look at the settings. Please edit the first post on this ticket and include your WP logins in the designated Sensitive Data section.
It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:
https://wordpress.org/plugins/temporary-login-without-password/
You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):
https://wordpress.org/plugins/wp-security-audit-log/
We really look forward to helping you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hello,
I checked your robots.txt in Rank math > General Settings > robots.txt and via https://yourdomain.com/robots.txt and I am seeing only the lines below:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://yourdomain.com/sitemap_index.xml
User-agent: *
Disallow: /*blackhole
Disallow: /?blackhole
I also tested the 404 monitor by testing a non-existing URL: https://yourDomain.com/robo and it was logged successfully. Could you please record (use https://www.loom.com) and share a screencast in the sensitive section.
Looking forward to helping you. Thank you.
​​​​​​
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hello,
Could you please check now?
To determine if this issue is originating from Rank Math or from your server, I have created a physical robots.txt file in your root folder and now it is being served by your server and not Rank Math.
If the issue persists, then it is an issue with your server and this issue is not originating from Rank Math.
If not, please let us know so we can get our dev. team involved into getting to the core of this issue.
Thank you.
Hi, I can confirm the issue has not persisted after a physical robots.txt file was created. The issue is therefore with the plugin itself.
Hello,
We will be taking a closer look at this.
Thank you for your feedback.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
