Hello,
Thank you for contacting Rank Math and sorry for any inconvenience that might have been caused due to that.
I have checked the URLs you shared and they seem to be set to index. I also checked your sitemap too and the URLs are also included in the sitemap.
So, could you please delete the old sitemap from Google Search Console and resubmit the main sitemap sitemap_index.xml to Google Search Console. Post that, give some time to Google so that it can crawl the URLs.
Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.
Looking forward to helping you. Thank you.
I do not think it is related with sitemap clearing. I added sitemap few days back only through rankmath support desk. The concern is reported by ahref, semrush too. So there is issue and I am suspecting it to be related with rank math.
I have one query about rankmath. I saw that our team has set couple of redirects in rankmath. They added the source url (which must be redirected) but the target url is left blank (rankmath does not makes it mandatory as where to redirect the url). So can this also results into 404 as redirct rule is valid but not configured correctly(as target url not given in rankmath) ?
I also see if I change the user agent as Google bot in chrome than page throws 404 error. I can see that error on page.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/96.0.4664.110 Safari/537.36
Hello,
To know what is causing the 404 errors on your site, you can enable the advanced monitor by navigating to WordPress Dashboard > Rank Math > General Settings > 404 Monitor > Enable the advanced mode
To know more about the advanced mode on 404 monitor, please check the link from our Knowledge Base article below:
https://rankmath.com/kb/monitor-404-errors/#advanced-404-monitor
Here is an article that will help you in understanding on how to fix the 404 errors:
https://rankmath.com/kb/fix-404-errors/
Please let us know how that goes. Looking forward to helping you further.
Thank you
Thanks but it looks to be a bug in Rankmath. As soon I am deactiavting the redirection rule then I am not getting 404 error on use of google bot as user agent.
You did not mention anything about the concern reported with redirection rules. Let me reiterate it.
Is it the behaviour of rank math to stop bots (throw 404 error) if there is a page redirection rule added where the rule contains only redirect “from” but does not have redirect “to” field set ?
To me it can be the reason why google bot is not able to find the page (atleast from the user agent test it looks to be a reason).
I will enable advance mode but I request rankmath support desk person to look into the concern thoroughly as this looks to be a bug in rankmath(high probability) and in this case you need to fix it asap.
Hello,
I investigated the pages and as you mentioned they’re properly loading on browsers. We only include the existing pages in the sitemaps and hence these pages are also present there.
However, I tested the URLs with https://httpstatus.io/ and I was able to replicate the issue. The URLs are returning 410 status codes. You can have a look at the screenshot in the sensitive data section of this ticket.
Could you please confirm if the redirection that you’ve created is for the 410 (Content Deleted) Maintenance Code? https://rankmath.com/kb/redirect-types/#410-content-deleted
If yes, then you should remove those redirects to fix the issue.
Regarding your question about the target URL in the redirections, the target URL is required for Redirection types only (301, 302, 307) and not for Maintenance codes (410, 451).
If you’ll leave the target URL field empty in a Redirection type redirect, then the empty field error would be shown.
Hope this helps.
Thank you.
Thanks Team. I have checked and I can see that the page redirect is set as 410 and hence “to” field is empty.I also understand that google and other bot will respect the 410 set on the page and they may show a 404 error.
I wanted to know if a user browse the page then he is not shown the 404 or 410 error ? So will user still see the page but bot cannot ?
Thanks
Hello,
When you set a page to 410 Maintenance code or if the page’s URL returns 404 status, usually, the page’s contents are not accessible.
In your case, the issue seems to be coming from somewhere else as when I previously tested the pages, they were returning 410 status, whereas I tested them again and now they’re returning 404s. You can have a look at the screenshot in the sensitive data section of this ticket.
The pages are still accessible from the browser.
Could you please temporarily disable the redirections for the URLs and then test the URLs with https://httpstatus.io/ and see if they return 200 (OK) status?
Looking forward to hearing back from you.
Thank you.
Hi, Here is the analysis(same as shared before)
1. The page is accessible from browser (does not matter I enable the redirection or not). This means if redirection 410 is enabled then “user” is still able to see the page in browser and rankmath is not showing 404 or 410. Thats the bug in rankmath (?).
2. The page throws 404 for bot if redirection is enabled. The page throws 404 in browser if I set user agent as google bot via chrome web inspecter. This means if redirection 410 is enabled bot will get 404 error
3. If redirection is disabled then bot (via user agent) does see the page and there is no 404 error or 410 (as redirection is disabled).
4. Moreover I think the the 410 redirection rule page are present in sitemap too and rankmath is not removing these page from sitemap.
Thanks,
Amit
Hello,
If a URL is set as 410 then it shouldn’t be accessible to users. We might need to take a closer look at the settings. Please edit the first post on this ticket and include your WordPress logins in the designated Sensitive Data section.
Please do take a complete backup of your website before sharing the information with us.
It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:
https://wordpress.org/plugins/temporary-login-without-password/
You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):
https://wordpress.org/plugins/wp-security-audit-log/
We really look forward to helping you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hi,
I have shared the details. Kindly have a look and let me know the reason of page visibility for users despite 410.
Thanks,
Amit
Hello,
I logged into your website’s admin area and upon investigating, found that if anyone among Elementor or the Hello Elementor theme is enabled, the page shows the data. If they’re disabled, it properly takes to the Not Found page. The 410 status is present in the console at all times.
Please have a look at the screencast in the sensitive data section of this ticket.
I tried to replicate this on my end but it seems to be working fine.
In this case, please get in touch with Elementor support as they might already have a solution for the issue and they would be in a better position to assist you.
Hope this helps.
Thank you.
ok. As per you the issue is in elementor. As per you if elementor plugin is enabled then the issue is appearing? Am I correct but I am still not sure(nor convinced) as why disabling the rank math redirection make page works for google bot even if the elementor plugin is enabled.
Hello,
Yes, you can have a look at the screencast I shared in the sensitive data section with the heading “Screencast (Elementor issue):”.
If you disable the redirection, the 410 status is removed from the page (even from the console) and this is the reason the page works for Google. Assuming you do not want the page to be accessible, as shown in my screencast if the “Elementor” plugin and “Hello Elementor” theme are not enabled, the page’s content does not load.
As I mentioned in my previous reply, I tested Elementor with the 410 redirection on my local installation but was unable to replicate the issue. In this case, it would be best if you get in touch with Elementor support as they would be able to assist you with the issue.
Hope this helps.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
