Hello,
Thank you for contacting Rank Math and bringing your concern to our attention. I’m sorry for any inconvenience this issue may have caused you.
I’ve checked your site, and it seems that they are already set to index. However, upon checking your robots.txt, it seems that you are still not allowing bots to crawl your site.
This kind of robots.txt prevents your site from getting indexed on the search results pages. To fix this, head over to your WordPress Dashboard > Settings > Reading and disable/uncheck this option
Once done, your robots.txt should be the default like this image.
I hope that helps.
Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.
Hi, thank you for your quick response!
I’ve checked my website but found that i’ve never click on the search engine visibility part.
And sorry i don’t know where can i find the image you sent to me.
Not sure if i check on the wrong place or not
Thank you so much
Hello,
Thank you for the update.
Since the Search engine visibility option is already disabled on your site, please check if you have a static robots.txt file in your website’s root directory. You can FTP to your server and delete that file as Rank Math automatically creates a virtual robots.txt to your site. Please get in touch with your web host for assistance to access your website files.
Moreover, your robots.txt should only contain this default rules/lines:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://pingpinglee464666326.wpcomstaging.com/sitemap_index.xml
You can edit and update your robots.txt in Rank Math > General Settings > Edit robots.txt. Here’s a link for more information:
https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/
Please note that some options are only available when you enable the Advanced Mode in Rank Math. If it’s not showing, you must have chosen to use the Easy Mode. Please follow the steps in this link to switch to Advanced Mode:
https://rankmath.com/kb/advanced-mode/
I hope that helps.
Thank you.
Hi, thanks for your reply.
I’ve found the robots.txt place and finished the settings.
So if the user agent appears like this: User-agent:*, it means google can’t crawl to my website? So if i changed it to : User-agent: googlebot, then my website will be allowed to be crawled right?
Not sure if i’m right or not, really a rookie in SEO lol.
Thank you so much.
Hello,
User-agent:* means that any web crawler can crawl your website.
User-agent: Googlebot means that only Googlebot will be able to crawl the website.
I hope this clarifies your query.
Let us know if you have any other questions.
We are here to help you.
Hello,
I have updated the sensitive data as requested. Can you please check further?
Thank you.
Hello,
I’ve checked your robots.txt, and it seems that it is still set to Disallow: /.
If you have already updated your robots.txt using Rank Math, please check your website files if there’s an existing static robots.txt file. You can get in touch with your web host if you’re unsure how to check your website’s root directory.
If there is a file, please delete that as Rank Math’s robots.txt is virtually created.
Once done, clear your website’s cache and check if the correct rules you have applied using Rank Math are already showing in your robots.txt.
I hope that helps.
Thank you.
Thanks for your reply
But sorry i still don’t understand how to correct the error of my web:(
I work on the web all by myself so~~
Is it ok to show me the steps and pics or what…
Cause i really have a hard time doing the setting.
Thank you so much
Hello,
If it’s more convenient for you, we can take a closer look at your website.
Please edit the first post on this ticket and include your WordPress & FTP logins in the designated Sensitive Data section.
Please do take a complete backup of your website before sharing the information with us.
It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:
https://wordpress.org/plugins/temporary-login-without-password/
You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):
https://wordpress.org/plugins/wp-security-audit-log/
We really look forward to helping you.
Thank you.
Hi
I click into the first pulgin and i downloaded, is that right?(below is the link)
[moved to sensitive data section]
And where is the ticket part? i click in the Woocommerce but didn’t see the same pic as yours?
Thank you.
Hello,
It looks like the link you have shared doesn’t seem to be working. Could you please check the link and share the updated one? You can also share the credentials if you are not sure about the login link.
Looking forward to helping you. Thank you.
Hi, thanks for your reply
I’ve downloaded the pulgin of temporary login but can;t find where to generate a URL for your team to access
What is the problem?
Can you please share the steps with pics to make me more clear?
Sorry and thanks for your help
Hello,
Please refer to this video on how to create a temporary login. https://youtu.be/n_VAKj04-l8?t=23
Looking forward to helping you on this one.
Hi
[moved to sensitive data section]
I just generated a link for the temporary log in, please check if it’s fine to access.
And the help i need is to set my SEO right to make sure if my website can be searched on GOOGLE.(I’ve tried so many times but still can’t solve the problem)
Thank you so much for your help.
Hello,
It looks like you have physical robots.txt on your WordPress root directory, which is overwriting Rank Math’s virtual one. To modify it, Could you please delete that robots.txt file from your server? If you are unsure, you can ask your host to do that for you, or You can share the FTP details with us so that we can look into it.
Looking forward to helping you. Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
