Hello,
We apologize for the inconvenience you’re facing with the robots.txt file not getting updated on your browser. We understand the frustration this may cause and we appreciate your patience.
To address this issue, we recommend checking if the robots.txt file is being cached by your browser or any content delivery network (CDN) like Cloudflare. Rank Math does not create a physical robots.txt file on your site. Instead, the file is created and displayed when a user visits your robots.txt URL. So you can safely ignore this error if your robots.txt file is available at https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/
If the issue persists, you can try adding the below rule to your robots.txt file and test it again in PageSpeed Insights:
Disallow: /cdn-cgi/
If you need further assistance, please feel free to reply to this ticket, and a human agent from the Rank Math Support team will be happy to help you.
Thank you for reaching out to us.
Sources:
Common robots.txt Issues & How to Fix Them » Rank Math
How to Fix the ‘Blocked Due to Other 4xx Issue’ Error » Rank Math
This is an automated reply from Rank Math Support Bot. If you need further assistance, please reply to this topic and a human support agent will reply to you shortly.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
