Hello,
Thank you for contacting Rank Math and bringing your concern to our attention. I’m sorry for any inconvenience this issue may have caused you.
If those pages are already set to noindex, they should be automatically removed from your sitemap. Could you please share the sitemap URL so we can check as well?
Meanwhile, for initial troubleshooting, please follow these steps to flush your sitemap cache:
1. Flush the Sitemap cache by following this video screencast:
https://i.rankmath.com/pipRDp
2. Exclude the Sitemap files of the Rank Math plugin in your caching plugin (if not yet). The cache could be via a plugin or from the server. For plugins or Cloudflare, please follow this article:
https://rankmath.com/kb/exclude-sitemaps-from-caching/
Looking forward to helping you.
Thank you.
Hi Reinelle,
Our sitemap can be found here: https://fiks.nl/sitemap_index.xml
I will try both tips.
Thank you.
Will get back to you when this didn’t solve it.
Best,
Niels
I actually found something that might cause it. Our theme makes sure these jobs are noindex when they are marked as filled, however, this does not communicate properly to Rank Math.
This job is filled and Google says it’s NoIndex, however, in the WP editor Rank Math says it’s Index, hence it appears in the sitemap: https://fiks.nl/en/vacancy/elanza-stage-community-manager-operations-bij-healthtech-startup-utrecht-5938/
Is there something I could tweak in Rank Math settings or should I write some custom code to make sure Rank Math index status will change whenever the job post status changes to “filled”?
Thanks!
Hello,
I can see the issue that the plugin you are using to manage the job posts is also adding a separate robots meta as you can see in my screenshot below:
I suggest checking this one out with the plugin/theme that handles your job posts as they are adding duplicate robots meta. If you can manage to remove the duplicate robots meta, you can then set the robots in Rank Math to noindex and avoid further conflicts.
Another option, if you wish to retain the noindex coming from the plugin/theme that manages your job posts, you can force Rank Math to clear robots meta by applying this filter code to your theme’s functions.php file:
/**
* Rank Math: filter the robots meta data.
*
* @param array $robots The meta robots directives.
*/
add_filter( 'rank_math/frontend/robots', function( $robots ) {
unset( $robots['index'] );
unset( $robots['follow'] );
return $robots;
});
Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.
Thanks for your reply. So I modified the code a bit so we’ll make sure this only happens for the filled job posts on our site. We use Google for Jobs and it would be very very bad if all of a sudden all jobs are noindex.
I’m a php noob, so any tips on my code below would be greatly appreciated. 🙂
add_filter( 'rank_math/frontend/robots', function( $robots ) {
if(get_post_type() == 'job_listing' && get_post_meta( get_the_ID(), '_filled', true )) {
unset( $robots['index'] );
unset( $robots['follow'] );
}
return $robots;
});
So I am testing the above locally and it seems to work. One problem I am running into: the filled jobs stay in the sitemap. I flushed the sitemap cache like Reinelle suggested locally, but I can still find these filled jobs in my sitemap. They are set to noindex by my theme and they do not contain robots from rank math. So my problem is still not solved right?
image: https://www.dropbox.com/s/ubs3372jz2tuch0/Xnapper-2022-06-28-15.05.41.png?dl=0
Hello,
If the noindex URLs are present in the sitemaps, yes, the issue is not fixed.
Please replace the previous code with the below one:
add_filter( 'rank_math/frontend/robots', function( $robots ) {
if(get_post_type() == 'job_listing' && get_post_meta( get_the_ID(), '_filled', true )) {
$robots["index"] = 'noindex';
$robots["follow"] = 'nofollow';
}
return $robots;
});
In this solution, we suggest you ask the theme developers to entirely remove their robots meta tag as Rank Math’s robots meta tag will do the needful.
Once done, clear the caches following the steps mentioned by my colleague and see if that helps.
Let us know how that goes.
Thank you.
Thank you Prabhat!
The thing is, we cannot ask our theme developers to remove this, as they will not do this. This would mean that we have double values of our robots meta, is this a problem, but both will say noindex in this case. Btw, why should we also add nofollow?
Thank you!
Hello,
The thing is, we cannot ask our theme developers to remove this, as they will not do this.
They do not need to update the theme, you can specifically ask for a code snippet for your website to disable the tag.
This would mean that we have double values of our robots meta, is this a problem, but both will say noindex in this case.
To allow Rank Math to detect the robots meta status of a post, the robots meta tag should be generated by Rank Math itself. Having duplicate tags is not recommended.
Btw, why should we also add nofollow?
You can remove nofollow if you do not want it.
One more approach that can be followed is by modifying and using this filter: https://rankmath.com/kb/filters-hooks-api-developer/#filter-sitemap-item
This filter filters the URLs before they’re added to the sitemap, irrespective of their robots meta status.
Hope that helps.
Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
