I am very concerned about this – I thought I had made a mistake. I went into the RankMath settings and turned off ALL instant indexing and saved. But when i checked, this had been overriden and Instant Indexing is now automatically sending out pages from my development site for indexing which is a massive massive problem as i am sure you appreciate. Please respond urgently.
I have now managed to turn off this module – but still very upset that the settings in there did not save, while it was active…. something needs to be done about that! And at the very least, either a warning on the module in the dashboard – or a switch to enable those working with RankMath in staging environments, to be able to disable it permanently for obvious reasons.
Now I need to know whether the over-arching block I had earlier set in place for our staging and development subdomains, with our ISP, will be overriden by your Instant Indexing – or whether that will still continue to block Google from those subdomains. If you can let me know asap please so I get some sleep tonight!
Also, I checked my settings – I was amazed to see that despite setting no index in the global settings, this had not applied to all pages or posts or types of posts consistently. I tried to use the bulk edit facility to correct these and that doesn’t work.
It’s very concerning that your plug in can override noindex settings applied both at wordpress (read) and theme (page) level, but also its own global meta settings. And having taking every precaution I possibly can to prevent staging sites from being indexed, your module has completely compromised that. To add insult to injury, I now find i have to manually correct the indexing on each page that has been affected.
Hello,
Thank you for contacting Rank Math for help disabling Instant Indexing on a development site, and sorry for the inconvenience and frustration you went through troubleshooting the issue. Thank you also for the detailed documentation.
Please check your Google Search Console index coverage report to verify that the submitted URLs were indeed handled by GSC and what their status is.
Please note that Rank Math cannot override x-robots-tag which is how development server noindex directives are usually assigned. The default priority for Rank Math robots meta is page level > post type level (if set to custom) > global meta. Unless you had manually set the individual pages to index and the default setting is noindex the pages should not be submitted whenever an instant indexing API request is submitted and the individual requests will not be indexed because they are set to noindex.
If you wish for us to investigate further, please edit the first post on this ticket and include your WordPress & FTP logins in the designated Sensitive Data section.
Please do take a complete backup of your website before sharing the information with us.
It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:
https://wordpress.org/plugins/temporary-login-without-password/
You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):
https://wordpress.org/plugins/wp-security-audit-log/
We really look forward to helping you.
Nigel, thank you, a very helpful reply which has gone some considerable way to reassuring me. I checked in google mobile friendly tool and the staging/development domain URLs were not accessible. Phew! The robots txt is both subdomains was set to “user agent disallowed” server side…. so i am very thankful I arranged that as it undoubtedly prevented what would have been a big problem!
Thank you for confirming the order of precedence – that was not clear from your documentation nor a reply I had from Bhanu!
1) I strongly suggest RankMath introduce a top level switch that allows people who need to install it on staging domains/subdomains to be able to completely override any possibility of submitting pages to Google by accident – either through activating the module in the dashboard without an upfront idea of the risks, or, as in my case, because the mouse hit the wrong link. For working in staging environments you absolutely have to install RankMath there – otherwise migration will not work. SO not installing the plug in in staging environments is not an option!
2) It would be advisable to have this as part of a security/index module in settings which also encompasses the order of precedence. This prompt needs to be one of the first things that happens after RankMath is installed IMO. The information is buried deep in your documentation, or, as in this case, is not quite specific enough.
3) It took me a long time to go through 3 sites to make sure all the staging items were noindex – at post level. It was not possible to do this in bulk. There has to be a better way…..?
4) I then had to check my live site where as a result of (3) I then had live pages that were set to noindex but which obviously needed to be indexable. Again, this took a lot of time. There needs to be a way to do this in bulk.
5) Now that part is done, I still have more work to do once the situation settles down. Anything that had to be set to noindex to counter Google needs to be reverted on Staging in order that the SEO settings can be passed through to the live site. If this is not done it means every time the development is migrated, a lot of time is needed to do this making site updates noticeably long – not ideal, as I am sure you understand.
Developers need to be sure that as they configure RankMath settings on staging sites for test purposes, that there is NO possibilty of inadvertently submitting those pages to Google accidentally.
RankMath is a superb product but when this happened I seriously considered abandoning it in favour of Yoast. You offer a lot for a free product, which is very much appreciated. But in view of what I went through, I hope Bhanu will listen to these product suggestions as I think it would make everyone’s life a lot easier.
Please pass him my message. I could not find the thread where I corresponded with him and do not wish him to think me impolite. I am glad he replied… and even more glad that you have nailed the issue. Kudos!
Thank Nigel, tell Bhanu you deserve a pay rise, or a box of choccies at least!
Hello,
Glad that helped.
We have forwarded your suggestions to the development team and they will check and consider them. Also, you can set your whole staging site to Noindex by enabling the Search Engine Visibility option in WP Dashboard > Settings > Reading.
If you do have another question in the future, please feel free to create a new forum topic, and it will be our pleasure to assist you again.
Thank you.
