About an indexing situation

#554006
  • Resolved Hodo
    Rank Math free

    Hello there.
    I am facing some problems:
    Both Google and Bing refuse to index the entire site, not even half of it (approx. 15k products).
    In the audits done online with free tools, the results are very contradictory. Some say that it is a problem of canonical, nonindex, redirect 301, http,there is no ga4,…others see them as normal.
    I personally do not find any of these.
    The site spent some time in the redesign, with the option “discourage search engines…”
    Either there is a conflict or I did something wrong.
    I would appreciate your help, please, because after a lot of work I seem tired, and maybe I don’t see simple and logical issues.
    Before the redesign, I had 300 products, which were indexed without problems, very well and quickly, RM helping me to quickly get where I wanted,…now…
    Thank you.

Viewing 15 replies - 1 through 15 (of 16 total)
  • Olivia
    Rank Math free

    Hello,

    It sounds like you are having some trouble with getting your website indexed by search engines. Here are some suggestions on what you can do:

    Make sure your robots.txt file is not blocking search engines from crawling your site. You can check this by going to yoursite.com/robots.txt and making sure there are no disallow rules for your pages.

    Check your site’s meta tags to ensure that you are not using a noindex tag. This tag will tell search engines not to index your pages.

    Use Google Search Console and Bing Webmaster Tools to identify any technical SEO issues that may be preventing indexing.

    Make sure your site has a clear site structure with easy-to-navigate menus and a sitemap that can be submitted to search engines.

    Ensure that your site is using HTTPS, as it is now a ranking factor for Google.

    Check your site’s mobile-friendliness, as mobile optimization is important for both user experience and SEO.

    Finally, be patient. It can take time for search engines to index a large number of pages, particularly if they have not been updated recently.

    I hope these suggestions help you get your site indexed. Good luck!

    Best Regards
    Olivia

    Hello,

    Thank you for contacting Rank Math and bringing your concern to our attention. I’m sorry for any inconvenience this issue may have caused you.

    @Olivia We appreciate you sharing those details about this issue.

    @Hodo I’ve checked your site, and it is already set to index as well as your products. Also, your sitemaps and robots.txt are working fine.

    In this case, please check your Google Search Console coverage section to determine any issues reported on your pages:
    https://support.artlogic.net/hc/en-gb/articles/360016851260-Google-Search-Console-errors-and-coverage-issues

    Looking forward to helping you.

    Thank you.

    Hodo
    Rank Math free

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    Hodo
    Rank Math free

    @Olivia Thank you very much for the information and involvement. I have already paid attention to these.

    @Reinelle According to gsc it seems to be a structural data issue “offers”, “review” or “aggregateRating”
    But for “review” or “aggregateRating” it should only be an endorsement.
    And for “offers”, I put $$ in the price.
    In my opinion, as a conclusion, it seems that in each category there are some products, which they do not like and because of them they do not continue indexing.
    All products are imported from the same feed, automatically.

    Let’s take the example given in sensitive data
    After that,
    I checked those products with other online audit tools
    I rechecked the structure.
    I put articles for pages.
    I also tested a snippet, so that page archives can be removed from the schema.

    
    function wc_remove_product_schema_product_archive() {
    	remove_action( 'woocommerce_shop_loop', array( WC()->structured_data, 'generate_product_data' ), 10, 0 );
    }
    add_action( 'woocommerce_init', 'wc_remove_product_schema_product_archive' );
    

    I cleaned the database, emptied all caches and transients, gave a recount,…
    I checked the respective products with different seo tools.
    Everything seems to be within the parameters.
    I don’t understand where the problem is…. The indexing started well and suddenly stopped in both Google and Bing

    Hello,

    The link you shared was not the actual product page, this was a product category archive page and which is why it shouldn’t contain the product schema in it.

    Please note that Google recommends disabling the schema to your product category archives. The Product Schema is only for the actual product, and not for anything else. You can check this link for more information:
    https://developers.google.com/search/docs/data-types/product#guidelines

    We’ve enabled the Remove Snippet Data option from your WordPress Dashboard > Rank Math > Titles & Meta > Att. Categories and now the page is validating fine on Rich Results Test. Please check the test result URL in the sensitive data section for reference.

    Hope that helps and please don’t hesitate to let us know if you have any other questions.
     
    Thank you.

    Hodo
    Rank Math free

    That’s right, that’s why I also tried with codes (carelessness and my fatigue). Thank you.
    Now the GSC is accepting reassessment for these.
    Anyway, do you see a problem somewhere with the “noindex” directive in a meta tag or HTTP header?
    In all my audits, nothing like this comes out.
    I did not find mixed content, duplicates, or noindex.

    Hodo
    Rank Math free

    I think I found the problem with contentking app help (can you confirm?), but I don’t know how to solve it, preferably without redirection:

    Present on non-canonical variants
    There is a robots.txt file available on 1 non-canonical domain variant.
    URL http://_____________/robots.txt
    Only the canonical domain should have a robots.txt file.

    I opened a few tickets before I launched the site on the topic of canonical.

    Hodo
    Rank Math free

    I also did the redirection but it doesn’t seem to have any effect.
    Anyway, I’ll wait a few days to see what Google says and come back.

    Hello,

    We can confirm from our end that all your domain variations are now redirecting to the final version, the same goes for your robots.txt file path. This shouldn’t cause any canonical issues now.

    Do let us know how that goes. We are here to help you with your issue.

    Thank you.

    Hodo
    Rank Math free

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    Hello,

    The sensitive data section only contains your website login information.

    Could you please confirm if you still have questions about this topic?

    Meanwhile, I’ve attached a screenshot for your robots.txt that the other versions are already redirecting properly.

    Looking forward to helping you.

    Thank you.

    Hodo
    Rank Math free

    Hi.
    Not indexing yet. Both Google and Bing. Blocked in the same state as when opening the ticket.

    In seo audits, with all the free tools, I do pretty well, even very well for start(even your analyzer says so).
    Google would have no reason not to index. If the content is not of “high quality” at this moment(but it is not the case), it should index and put me in the queue.

    Bing says this: “The page contains the “noindex” directive in a meta tag or HTTP header. If you want this page to be indexed and appear in Bing search results, you must remove the HTTP tag or header from the page.

    I searched and did not find “noindex”. And anyway, everything is redirected to http htpps, as you can see.

    So I searched during this time where the problem could be.
    I think there are two problems:
    1. There are many duplicates/redirections from my next action: I think (actually, certainly at that time), that I had a conflict between the theme, the optimization program and RM. Then, I disabled RM and the optimization program for an interval of about 30-40 minutes. At that moment, many were indexed with “base” (I used the “remove base” option in RM for everything). But let’s say that this is not so serious. Google should index the correct links and these a moment should disappear or if not I tell him to “Remove outdated content” .?!

    2. Structural data. Actually here (I think) is the problem.
    In the schema.org test there is no warning, also in no other online tool, it does not say anything negative about structural data. Instead, I receive (recent) results from Google Rich Test:

    Structured data with syntax errors detected
    Invalid items are not eligible for rich Google Search results.
    Invalid top-level element ‘null’

    I honestly don’t know what to do anymore.
    Thank you.

    Hello,

    1. Please note that when the Remove Base or category base is enabled in Rank Math, the previous URLs are automatically redirected to the new permalink structure.

    Also, your site’s already indexed on Google. If there are any issues reported on your URLs that prevent them from being indexed, then you should see it in the coverage section report.

    Some of your URLs are already appearing in the Bing search results pages as well. We have already verified that your site has no noindex meta tag upon checking.

    2. I’ve checked your site using Health Check Plugin, and it seems that you have a snippet created which adds the script type="application/ld+json">null in the source code that causes the issue on your homepage. Please refer to the screencast attached in the sensitive data section.

    In this case, you may need to check the snippets causing this issue to fix the error on your site.

    Looking forward to helping you.

    Thank you.

    Hodo
    Rank Math free

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    Hodo
    Rank Math free

    Hi.
    It is.
    I think you found the source of the problem as you can see in the attached codeshare.
    In a code that adds the brands to the scheme.
    And I was looking elsewhere for that “null”, as it appears in the source of the page.
    Moved the $schema variable inside the add_brand_schema() function to avoid global variable usage.
    Now everything seems to be normal, I’m waiting for the “google bots” to do their job.
    Thank you very much for the support.

Viewing 15 replies - 1 through 15 (of 16 total)

The ticket ‘About an indexing situation’ is closed to new replies.