ROBOTS.TXT Issue

#140853
  • Resolved Jwlr Addicts

    4 years, 10 months ago

    Rank Math free

    Hi Guys,

    My question is about the robots file generated by Rankmath.

    I’m using AliDropship plugin and their Robots file looks different than one from the Rankmath’s.

    See below samples of both.

    This is one generated by alidropship:
    User-agent: *
    Disallow: /cgi-bin
    Allow: /wp-includes/js
    Allow: /wp-includes/css
    Disallow: /wp-includes
    Allow: /wp-content/plugins/alids/assets
    Allow: /wp-content/plugins/alids/src
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /trackback
    Disallow: */trackback
    Disallow: //trackback
    Disallow: //feed/*/
    Disallow: */feed
    Allow: /wp-admin/admin-ajax.php
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /*?cur=*
    Disallow: /*?*cur=*
    Sitemap: https://mywebsite.com/sitemap.xml
    Host: https://mywebsite.com

    Rankmath Robots.txt file

    User-agent: *
    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php
    Sitemap: https://mywebsite.com/sitemap_index.xml

    My question is what to do? should I copy the content of Alidropship file to Rankmath file.

    I just don’t want to harm my website.

    Please kindly advise.

    Regards,
    Sergei

Viewing 5 replies - 1 through 5 (of 5 total)
  • Danial

    4 years, 10 months ago

    Rank Math business

    Hello,

    Thank you for contacting Rank Math, and sorry for any inconvenience that might have been caused due to that.

    For Rank Math’s recommended robots.txt, you need to have /wp-admin/ to be disallowed and /wp-admin/admin-ajax.php to be allowed.

    While sitemap needs to be pointing to https://domain.com/sitemap_index.xml as Rank Math generates its sitemap there.

    I hope this helps you with your issue.

    Jwlr Addicts

    4 years, 10 months ago

    Rank Math free

    Thanks Danial,

    I have another question about sitemap settings for “Products”
    “If you use WooCommerce to manage an online store, the Products tab will be relevant to you as it contains all the settings that affect the sitemap with respect to products. Here is a look at all the settings available in this tab”

    In case I’m not using Woocommerce on my online store, should I enable this?

    thanks again
    Sergei

    Michael Davis

    4 years, 10 months ago

    Rank Math free

    Hello,

    Thank you for the follow up.

    Would you please share a screenshot of which settings you are referring to? You can use https://imgbb.com for screenshots.

    If however you don’t use WooCommerce, you can ignore WooCommerce related settings. Would you also please share which plugin you are using for your store?

    Looking forward to helping you.

    Jwlr Addicts

    4 years, 10 months ago

    Rank Math free

    Hi,

    See below screenshot link

    https://ibb.co/z75tZ9b

    I’m using AliDropship plugin but not woocommerce version.

    Thanks
    Sergei

    Michael Davis

    4 years, 10 months ago

    Rank Math free

    Hello,

    If , as you said, you don’t have WooCommerce, the Product post type shown under sitemap is your store products.

    I would recommend to enable this option if you wish to have your products included in sitemap for indexing.

    The tabs in Rank math > Sitemap settings usually list all post types and pages on your site. In case of Products, they don’t have to be from WooCommerce.

    Hope this helps

    RankBot

    4 years, 9 months ago

    Hello,

    Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.

    If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.

    Thank you.

Viewing 5 replies - 1 through 5 (of 5 total)

The ticket ‘ROBOTS.TXT Issue’ is closed to new replies.