Hello,
The behavior of robots.txt with HTTP sometimes redirecting to the HTTPS version can vary based on your site’s configuration and server settings. Ideally, the robots.txt file should be accessible directly via HTTP without any redirection to the HTTPS version.
If you are experiencing redirection issues with your robots.txt file from HTTP to HTTPS, it might be due to server-side configurations or rules in your .htaccess file that force HTTPS redirection for all URLs. To ensure proper accessibility, you should check your server settings and configurations to allow direct access to the robots.txt file via HTTP without any redirection.
If you need further assistance in resolving this issue, please provide more details about your website setup, server type (Apache or Nginx), and any relevant configurations so we can assist you better.
Sources:
No Information is Available for This Page » Rank Math
Common robots.txt Issues & How to Fix Them » Rank Math
Advanced Tab in Rank Math SEO’s Meta Box
How to Fix the ‘Blocked Due to Other 4xx Issue’ Error » Rank Math
There may also be problems with sitemaps. Could you check if the robots.txt and sitemaps you generate create conflicts with Bluehost?
Bluehost doesn’t help me, as it’s your problem and they don’t take responsibility. I’ve been trying for a year and a half to make them understand that I have problems with the SERP, sitemaps and robots.txt.
Hello,
Thank you for your query and we are so sorry about the trouble this must have caused.
We checked your website’s robots.txt and all HTTP don’t redirect to HTTPS version. If you wish to redirect all HTTP version to HTTPS, this is something you can do by following this guide: https://www.semrush.com/blog/redirect-http-to-https/
As for your question about the conflicts, your sitemaps and robots.txt are loading just fine. Just make sure that your current URL variation (HTTPS and www or non-www) matches the URL you submitted to the Google Search Console.
Looking forward to helping you.
I don’t want them to redirect or at the very least, robots.txt files to have stable code. For a while the code is 200, then 301 and then 200. And so on.
Hello,
We’re not sure why it is behaving that way but this most likely stems from your SSL/HTTPS setup and not Rank Math. The plugin simply generates your virtual robots.txt file, it does not control the protocol it is served in – you would have to reach out to your host for help with that.
You could also use other options like the ReallySimpleSSL plugin to force your website to use the HTTPS protocol.
Thank you for your patience and cooperation
I’ve asked to hosting and added plugin Really Simple SSL too. Google support say is normal. For me isn’t so, because when i’ve lost HTTPS and breadcrumbs from Search Console, i’ve lost als keywords and pages on SERP.
But maybe i’ve found solution. I hope will be this.
Hello,
Let us know how it goes.
Meanwhile, if you have any other questions, please feel free to ask.
We are here to help you.
I won’t remember to write on this ticket. If it goes well, the games I have on the site will be added to the SERP. URLs are currently indexed, but excluded from the results.
Hello,
Can you please share some of the URLs that are indexed but excluded in SERP.
Also, please check if you are getting data in the performance report. If you are getting data in the performance report, then it means that your page is indexed in Google and probably not visible in SERP due to high competition.
We hope this helps. Please let us know if you have further questions or concerns.
Thank you.
https://imgur.com/a/Xt8FkHj
Every mine urls are on SERP, but not are ranked.
Bluehost has finally checked troubles with robots.txt and say that “Rank Math it’s too aggressive”. They mean that your plugin stress too thier servers.
Hello,
We are very proud to have built a fast-performing plugin and it’s one of the fastest SEO plugins on the market at the moment.
Saying that our plugin is very taxing for the server it’s something we take very seriously so if they have more information on the queries and functions that are performing poorly on that server we would gladly address them.
It’s also important to note, that the server resources and the number of plugins installed play a huge role in this.
Don’t hesitate to get in touch if you have any other questions.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hello,
Thank you for sharing but the information you shared will not help for the issue you’re referring to. If Bluehost support says our plugin is tasking to their servers, please ask them to deactivate other plugins and use a default theme – then share the specific queries and functions that are performing poorly.
Once we receive the information about the queries and function, we can investigate them on our end.
Thank you for choosing Rank Math
In the meantime I downgraded PHP from 8.2 to 8.0. Because many plugins have support up to 8.0
Hello,
Sure that’s fine. The minimum required PHP version to use Rank Math is currently set at PHP 7.4.
Let us know if you need help with anything else.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
