Editing Robots.txt

#35691
  • Resolved Michael Martin

    5 years ago

    Rank Math free

    Hi all,

    I’m attempting to edit robots.txt within RankMath, but am not able to enter any text in the input box. In the help file I can’t see anything that would explain why it’s not editable. The only thing mentioned is that if there is robots.txt in root directory, then any changes made will not apply. However, I can’t make any changes in the first place (and there is no robots.txt in my root directory in any case). Please could you advise?

    Thank you.

    Michael.

Viewing 9 replies - 1 through 9 (of 9 total)
  • Michael Davis

    4 years, 12 months ago

    Rank Math free

    Hello,

    Thank you for contacting Rank Math today.

    I have checked your site and the robots.txt file is editable as shown below:
    Robots

    Please note that the file is empty by default in order to allow the plugin to control the settings dynamically depending on how you have configured the robots meta of your pages.

    Looking forward to helping you. Thank you.

    ​​​​​​

    Michael Martin

    4 years, 12 months ago

    Rank Math free

    Hi Michael,

    This is what I see, and the field is not editable:

    https://www.dropbox.com/s/6s1jrbi8ulngqef/rankmath-robots-txt.jpg?dl=0

    Michael Davis

    4 years, 12 months ago

    Rank Math free

    Hi there,

    Thanks for the reply.

    In that status, you only need to set the cursor in the text box and start typing and then hit save.

    Let us know how this goes.

    Michael Martin

    4 years, 12 months ago

    Rank Math free

    Eh? I could have sworn I tried to add text in the input box….

    Are these settings recommended (the greyed out ones):

    User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://reclaimdesign.org/sitemap_index.xml
    Michael Davis

    4 years, 12 months ago

    Rank Math free

    Hi Michael,

    Thanks for the reply.

    Yeah you should keep it blank with the default settings since at most times custom robots settings will conflict with the ones set in the plugin.
    Rank Math will then update the robots.txt file dynamically.

    I hope this info helps. Thank you.

    Michael Martin

    4 years, 12 months ago

    Rank Math free

    Thanks Michael. Earlier I added this – can you confirm it’s OK please?

    User-Agent: *
Allow: /wp-admin/admin-ajax.php
Allow: /wp-content/uploads/
Disallow: /wp-admin/
Disallow: /wp-content/uploads/wpallimport/
Disallow: /wp-content/uploads/sucuri/
Disallow: /wp-content/uploads/screaming-frog/
Disallow: /wp-content/uploads/js/
Disallow: /wp-content/uploads/fonts/
Disallow: /wp-content/uploads/ao_ccss/
Disallow: /*?reviews-page=*
Disallow: /*?*reviews-page=*

Sitemap: https://reclaimdesign.org/sitemap_index.xml
    Michael Davis

    4 years, 12 months ago

    Rank Math free

    Hello,

    Thank you for contacting Rank Math today.

    It seems OK. Always keep in mind to only allow sections of your site you wish Search Engines to crawl and Disallow if you don’t wish to.

    Looking forward to helping you. Thank you.

    ​​​​​​

    Michael Martin

    4 years, 12 months ago

    Rank Math free

    Thanks Michael. As long as I’m not breaking anything 🙂

    Todd

    4 years, 12 months ago

    Rank Math free

    Hi Michael,

    It is always a good idea to test your changes:
    https://www.google.com/webmasters/tools/robots-testing-tool

    Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.

Viewing 9 replies - 1 through 9 (of 9 total)

The ticket ‘Editing Robots.txt’ is closed to new replies.