Hello,
Thank you for contacting Rank Math today.
You can dissalow all URLs that have /javascript/ in them to be excluded from search engines. To do that you can use the Robots.txt option:
https://rankmath.com/kb/general-settings/#edit-robotstxt
Let us know if that helps.
-
This reply was modified 5 years, 1 month ago by
Mili.
My understanding is that although Google and some others will obey robots.txt, many other sderach engines and bots will not respect the disallow so perhaps although this may cut down on the 404s, will not stop them
In this case I might use a redirect that for anything that includes /javascript/ as I can’t believe there is a legitimate reason for anyone, human visitor or bot, to be searching that.
Any thoughts?
Hello,
The reason the JS/CSS and fonts are logged in 404 Monitor is because those links don’t exist on your site and someone or the Search engine bots tried to access it. Please make sure those links don’t show up anywhere on the frontend in the page source. You can also check the Crawl errors in Dashboard >> Rank Math >> Search Console >> Crawl Errors to see if Google bot is crawling those URLs.
You can use some tool like Screaming Frog to find the broken links on your site. This tool is built specifically to behave just like search engine spiders and will, therefore, help you find a lot of issues.
Hope that helps. Thank you.
Hello,
Since we did not hear back from you for 15 days, we are assuming that you found the solution. We are closing this support ticket.
If you still need assistance or any other help, please feel free to open a new support ticket, and we will be more than happy to assist.
Thank you.
