Hello,
Thank you for contacting Rank Math today.
Rank Math will generate a virtual robots.txt file for your site and thus it is not recommended to create a static file since Rank Math updates this file dynamically depending on the robots setting that you configure on the Rank Math > Titles & Meta settings.
Looking forward to helping you. Thank you.
Hi Michael
thanks for your response
I know it and it’s wonderful
but how about this?
Disallow: /cgi-bin/
disallow: /wp-includes/
disallow: /amp/
disallow: /wp-admin/
disallow: /feed/
disallow: /page/
disallow: *?*
disallow: */?checkout=*
disallow: */?s=*
disallow: */search?q=*
disallow: */?*
disallow: *utm_*
disallow: */?lmxnuvmyq=*
disallow: */?DA=*
disallow: */?ypexbygeq=*
disallow: */?SD=*
disallow: */?paged=*
disallow: */?DD=*
Allow: *.js
allow: *.css
allow: /wp-admin/admin-ajax.php
Can I add them? what is this addresses?
thanks a lot
Hi Milad,
Thanks for the reply.
Before adding these directives on your robots file it is best to be sure that it does what you want. In the example above, the robots file would block indexing for all links that contain /page and all /amp pages
If you are sure that this is the desired behavior then you can head over to Rank Math > General Settings > Edit robots.txt file and add the directives.
I hope this info helps. Thank you.
Hi Michael
thanks for your response
which of the line above is popular?
I want more optimization for rank math
I search a lot about it. but I couldn’t find anything.
thanks a lot
Hello,
Thank you for contacting Rank Math today.
When you Disallow links it implies they wont be accessed or indexed by search engines. I would recommend you leave the robots.txt file as it is and only add Disallow/Allow if you are sure about it.
I hope this helps you. Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
