-
Now my suspicions that robots.txt is in the cache are well founded. I still see the old version of robots.txt. Instead another user sees the new one.
https://imgur.com/a/CpVWnnD
https://snipboard.io/7EoJvd.jpgI have cleared the plugin cache and Cloudflare cache but not the browser cache. Unfortunately I still see the problem.
I wish I could fix it without clearing the browser cache if it’s possible as this would fix the crawler issues.
-
Hello,
For the website : https://giocone.com
We can see the robots file here : https://giocone.com/robots.txt
Please refer to the screenshot below :
For the website : https://mrbivak.com
We can see the robots file here : https://mrbivak.com/robots.txt
Please refer to the screenshot below :
Could you please clarify what you expect to see and also share the expected robots file rules here as well for both the site?
If you want we can take a closer look at the settings, and for that please edit the first post on this ticket and include your WordPress & FTP logins in the designated Sensitive Data section.
Please do take a complete backup of your website before sharing the information with us.
It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:
https://wordpress.org/plugins/temporary-login-without-password/
You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):
https://wordpress.org/plugins/wp-security-audit-log/
We really look forward to helping you.
Thank you.
The ticket ‘The robots.txt is in the cache’ is closed to new replies.