rank_math_robots are causing sites to go down and significant performance issues

#850530
  • Resolved muhammadtvk

    1 year, 4 months ago

    Rank Math free

    Hi Team,

    I’m using both Rank Math Pro and Free versions. We have 171,105 WP posts on our sites. When I enable the sitemap, which is necessary for our site, it causes significant performance issues and the sites go down. Could you please address this issue as soon as possible?

    Rank Math SEO -> Version 1.0.224
    Rank Math SEO PRO ->Version 3.0.67

    PHP Configuration
    ———————
    PHP Memory Limit: 1024M
    Max input time: 150
    PHP time limit: 120
    PHP max input variables: 1000

    Below are the 2 queries keep hitting on the database –

    Query -> 1
    —————
    $where_filter = $this->do_filter( ‘sitemap/get_posts/where’, ”, $post_types );
    $sql = ”
    SELECT l.ID, post_title, post_content, post_name, post_parent, post_author, post_modified_gmt, post_date, post_date_gmt, post_type
    FROM (
    SELECT DISTINCT p.ID FROM {$wpdb->posts} as p
    {$join_filter}
    LEFT JOIN {$wpdb->postmeta} AS pm ON

    Query -> 2
    ———–

    $where_filter = $this->do_filter( ‘sitemap/post_count/where’, ”, $post_types );

    $sql = “SELECT COUNT( DISTINCT p.ID ) as count FROM {$wpdb->posts} as p
    {$join_filter}
    LEFT JOIN {$wpdb->postmeta} AS pm ON ( p.ID = pm.post_id AND pm.meta_key = ‘rank_math_robots’ )
    WHERE (
    ( pm.meta_key = ‘rank_math_robots’ AND pm.meta_value NOT LIKE ‘%noindex%’ ) OR
    pm.post_id IS NULL

Viewing 4 replies - 1 through 4 (of 4 total)
  • RankBot

    1 year, 4 months ago

    Hello,

    We apologize for the issues you are facing with your site performance when enabling the sitemap in Rank Math. Thank you for providing detailed information about your setup and the queries that are causing problems.

    Given that you have a large number of posts (171,105), it’s possible that the queries related to fetching posts for the sitemap are putting a strain on your database. Here are a few suggestions to help mitigate the performance issues:

    1. Check for Plugin Conflicts: Sometimes, other plugins can conflict with Rank Math. You can use the Health Check plugin to identify any potential conflicts.

    2. Optimize Database: Consider optimizing your database to improve performance. This can include cleaning up post revisions, spam comments, and transient options.

    3. Increase Server Resources: Although your PHP memory limit is set to 1024M, if your hosting environment allows, consider upgrading to a more powerful server or a dedicated hosting plan that can handle high traffic and large databases more efficiently.

    4. Caching Solutions: Implement caching solutions to reduce the load on your server. This can help serve cached versions of your pages and reduce database queries.

    5. Custom Code: If you are comfortable with coding, you might want to consider adding custom code to limit the number of posts fetched for the sitemap or optimize the queries further. However, this should be done carefully to avoid breaking functionality.

    6. Contact Hosting Provider: If the issue persists, it may be beneficial to reach out to your hosting provider for assistance, as they may have insights into server performance and optimizations specific to their environment.

    If these suggestions do not resolve the issue, please reply to this ticket, and a human agent from the Rank Math Support team will take over and assist you further.

    Thank you for your understanding, and we hope to resolve this issue promptly!

    Sources:
    Rank Math Sitemap Issues and Their Fixes » Rank Math

    Miguel

    1 year, 4 months ago

    Hello,

    Thank you for contacting support.

    For the sitemap generation to work correctly our plugin needs to check the robots meta settings for each of the posts/pages on the website to know which ones it should add to the sitemap.

    Since you have so many posts you need to make sure that you have sufficient CPU and memory to be able to handle this call from the plugin.

    Unfortunately, this is not something we can change because of the reasons mentioned above so you need to make sure that you have enough server resources to handle all the data you have on the website.

    Don’t hesitate to get in touch if you have any other questions.

    muhammadtvk

    1 year, 4 months ago

    Rank Math free

    Thanks Miguel,

    We have sufficient memory and CPU. Most of the time, it only uses 5 to 10%. The issue is that the database keeps loading due to the above-mentioned query. In the process list, those queries are stuck. Below are the memory and CPU details.

    AWS server
    db.r5.xlarge
    vCPU
    4
    RAM
    32 GB
    Mysql Aurora

    We believe Rank Math should be able to handle a CMS with a higher post count.

    The below-mentioned query takes 8 to 10 seconds, and we have hundreds of requests processing MySQL at the same time, which causes the site to go down.

    SELECT COUNT( DISTINCT p.ID ) AS COUNT FROM wp_posts AS p LEFT JOIN wp_postmeta AS pm ON ( p.ID = pm.post_id AND pm.meta_key = ‘rank_math_robots’ )

    SELECT l.ID, post_title, post_content, post_name, post_parent, post_author, post_modified_gmt, post_date, post_date_gmt, post_type FROM (SELECT DISTINCT p.ID FROM wp_posts AS p LEFT JOIN wp_postmeta AS pm ON ( p.ID = pm.post_id AND pm.meta_key = ‘rank_math_robots’ )

    If you could optimize this query or find an alternative solution, it would be great for sites with a higher post count.

    If you could provide any patch or temporary fix, it would be greatly appreciated, as our site has been down for the past couple of days and clients have escalated multiple times.

    Thanks for understanding..

    Miguel

    1 year, 4 months ago

    Hello,

    At the moment we don’t have a temporary patch or a changed query that could result in faster results. The query is done like this because we need to get the information for the sitemap generation to be accurate.

    You can try indexing the MySQL entries to speed up the queries using the following plugin: https://wordpress.org/plugins/index-wp-mysql-for-speed/

    Please make sure that you take a complete backup if you want to try that method.

    Don’t hesitate to get in touch if you have any other questions.

    RankBot

    1 year, 3 months ago

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 4 replies - 1 through 4 (of 4 total)

The ticket ‘rank_math_robots are causing sites to go down and significant performance issues’ is closed to new replies.