How to set a canonical or noindex of a job listing post for multilingual sites?

#528847
  • Resolved G
    Rank Math business

    Hi,

    I run a job board and we have 2 languages on our website. I have a problem where Translatepress plugin automatically creates a page for my extra language. Our main language is Dutch which has no language tag / and the other language is English which has the /en/ in the URL.

    This means Google often sees I have duplicate content and it will randomly assign which is the correct version, which means that often a Dutch text is actually indexed on our /en/ part of the site, I do not want this.

    In addition to this we use structured data for Google for Jobs and also here it will assign randomly whether it takes the / or /en/ version, without actually looking at the language on the page.

    I would love some advise on what is the best strategy SEO wise. I thought of these options:
    – Using canonicals from the incorrect version to the correct version to hint to Google which one to prioritize
    – Using a filter to noindex all job listings with Dutch text for their equivalent /en/ pages and vice versa.
    – Not generating schema markup for the incorrect version so Google does not import it in Google for Jobs

    Which of these options or a combination of these options would you recommend?

    Attached advise from a Google forum, which, to be fair, I didn’t really understand, but maybe you guys do.

    Thanks a lot! This has been a big problem for a long time and I would love to fix it.

Viewing 12 replies - 1 through 12 (of 12 total)
  • G
    Rank Math business
    add_filter( 'rank_math/frontend/robots', function( $robots ) {
        $post_id = get_the_ID();
        if (get_post_type($post_id) === 'job_listing') {
            $current_url = home_url(add_query_arg(array(), $wp->request));
            $terms = wp_get_post_terms($post_id, 'job_languages', array('fields' => 'names'));
    
            if (in_array('Nederlands', $terms) && strpos($current_url, '/en/') !== false) {
                $robots["index"] = 'noindex';
                $robots["follow"] = 'follow';
            } elseif (!in_array('Nederlands', $terms) && strpos($current_url, '/en/') === false) {
                $robots["index"] = 'noindex';
                $robots["follow"] = 'follow';
            }
        }
        return $robots;
    });

    I wrote this code to noindex the pages I want not to show up in Google. I was wondering whether I also should add a canonical to the correct version and whether I should also write code that takes the “wrong” URLs out of my sitemap. Thought I’d share it already, looking forward to your response!

    G
    Rank Math business

    So I kept improving my code. An SEO expert advised me to set the pages “noindex” and set a canonical as well. Right now I have the following code. I first tried using a Rank Math filter but this conflicted with my $robots filter, so I found this solution:

    function modify_rank_math_canonical() {
      $post_id = get_the_ID();
      if (get_post_type($post_id) === 'job_listing') {
          $current_url = home_url(add_query_arg(array(), $wp->request));
          $terms = wp_get_post_terms($post_id, 'job_languages', array('fields' => 'names'));
          $page_uri = get_page_uri($post);
          $home_url = 'https://' . $_SERVER['HTTP_HOST'];
          $full_dutch_slug = $home_url . '/vacature/' . $page_uri;
          $full_english_slug = $home_url . '/en/vacancy/' . $page_uri;
          if (in_array('Nederlands', $terms) && strpos($current_url, '/en/') !== false) {
              remove_action('wp_head', 'rank_math_canonical');
              echo '<link rel="canonical" href="' . esc_url($full_dutch_slug) . '" />' . "\n";
          } elseif (!in_array('Nederlands', $terms) && strpos($current_url, '/en/') === false) {
              remove_action('wp_head', 'rank_math_canonical');
              echo '<link rel="canonical" href="' . esc_url($full_english_slug) . '" />' . "\n";
          }
      }
    }
    add_action('wp_head', 'modify_rank_math_canonical', 11);
    
    add_filter( 'rank_math/frontend/robots', function( $robots ) {
      $post_id = get_the_ID();
      if ( get_post_type( $post_id ) === 'job_listing' ) {
          $current_url = home_url( add_query_arg( array(), $wp->request ) );
          $terms       = wp_get_post_terms( $post_id, 'job_languages', array( 'fields' => 'names' ) );
          if ( in_array( 'Nederlands', $terms ) && strpos( $current_url, '/en/' ) !== false ) {
              if ( isset( $robots['index'] ) && $robots['index'] === 'index' ) {
                  $robots['index']  = 'noindex';
              }
          } elseif ( !in_array( 'Nederlands', $terms ) && strpos( $current_url, '/en/' ) === false ) {
              if ( isset( $robots['index'] ) && $robots['index'] === 'index' ) {
                  $robots['index']  = 'noindex';
              }
          }
      }
      return $robots;
    });

    I think this is ok now and this could be well out of the scope from this support, but if you would be able to comment whether this solution is good use of your documentation, that would suffice. Thanks a lot!

    Nigel
    Rank Math business

    Hello,

    Thank you for contacting Rank Math for help with duplicate content issues on your multilingual site.

    I checked your website and found several pages that had identical content on both the Dutch and English version of your website. For example the following two posts are both in Dutch even though on should be in English:

    https://fiks.nl/vacature/dutch-boosting-group-traineeship-systems-engineering-en-systeemdenken-veenendaal-15828/
    and
    https://fiks.nl/en/vacancy/dutch-boosting-group-traineeship-systems-engineering-en-systeemdenken-veenendaal-15828/

    Setting the extra pages to noindex as in the code snippet you shared should work to remove the duplicate content warnings. You can also set canonicals for the correct versions. Ideally, you’d create translations for the duplicate pages that need translations, so you wouldn’t need to noindex any pages.

    Hope that helps. Please let us know if you have questions.

    G
    Rank Math business

    Hi,

    Unfortunately not yet resolved. The jobs with the above code stop appearing in Google for Jobs. I checked and a job posted a week ago was not in my sitemap yet. Now I just refreshed the sitemap by changing the amount of links that can be present in the sitemap, which updated it.

    Now I see the job https://fiks.nl/vacature/saleslift-studio-stage-sales-bij-ambitieuze-sales-developer-amsterdam-20693/ in my sitemap, but it also listed the noindex job:
    https://fiks.nl/en/vacancy/saleslift-studio-stage-sales-bij-ambitieuze-sales-developer-amsterdam-20693/

    Two questions:
    1. My sitemap does apparently not update itself, how do I fix this?
    2. Why does a noindex page end up in my sitemap and how do I get rid of it?

    Thanks!

    Hello,

    Thank you for your patience.

    If the changes are not reflected to your sitemaps and only refresh upon changing the links per sitemap, please exclude the Sitemap files of the Rank Math plugin in your caching plugin. The cache could be via a plugin or from the server. For plugins or Cloudflare, please follow this article:
    https://rankmath.com/kb/exclude-sitemaps-from-caching/

    If the issue still persists, please try adding this filter to your active theme’s functions.php file:

    add_filter( 'rank_math/sitemap/enable_caching', '__return_false');
    

    Let us know how this goes.

    Thank you.

    G
    Rank Math business

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    G
    Rank Math business

    Hi Reinelle,

    Thanks!

    Unfortunately it didn’t solve it. I added the snippet to my functions.php on our test environment.

    These entries are still in the sitemap:
    https://sandbox.fiks.nl/vacature/fiks-cloned-first-time-beuningen-20818/
    0 2023-02-28 10:06 +00:00
    https://sandbox.fiks.nl/en/vacancy/fiks-cloned-first-time-beuningen-20818/
    0 2023-02-28 10:06 +00:00

    As you can see only the first one is index, the second one is noindex, so it should not be in there right?

    I updated the permalinks and the number of pages for the sitemap, then it also did not work yet.

    In the sensitive data I added a login for you, so you can have a look if you want.

    Through the backend> theme editor > functions.php you can test code if you want. Don’t worry if you break the site, it’s a development site anyways.

    Thanks a lot!

    Nigel
    Rank Math business

    Hello,

    When I checked your staging site, both URLs you shared were set to noindex and were no longer in the sitemap. Please confirm if this is correct and if you need further assistance.

    Looking forward to helping you.

    G
    Rank Math business

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    G
    Rank Math business

    Apologies, the last jobs are both noindex for some reason… Something is setting everything I publish to noindex now. I will figure this out and report back when I have 2 jobs I can share.

    G
    Rank Math business

    My apologies, my team was testing things. I will make sure this will stay untouched:

    https://sandbox.fiks.nl/vacature/fiks-one-should-be-expired-beuningen-20849/
    https://sandbox.fiks.nl/en/vacancy/fiks-one-should-be-expired-beuningen-20849/

    Both show up in the sitemap, where the first one should not, as it’s set to noindex.

    Nigel
    Rank Math business

    Hello,

    Thank you for the update. Please let us know when you are ready to continue troubleshooting.

    Looking forward to helping you.

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 12 replies - 1 through 12 (of 12 total)

The ticket ‘How to set a canonical or noindex of a job listing post for multilingual sites?’ is closed to new replies.