Robots.txt – Error server (5xx)

Hello,

We apologize for the issue you are facing with your robots.txt file and its impact on your site’s indexing. The "Error server (5xx)" indicates that there is a server-side issue preventing Google from accessing your robots.txt file. This can happen for several reasons, including server misconfigurations or issues with your hosting provider.

Here are some steps you can take to troubleshoot this issue:

1. Check Server Status: Ensure that your web server is running properly and there are no outages or maintenance activities affecting it.
2. Access the robots.txt File: Try accessing your robots.txt file directly by visiting https://yourdomain.com/robots.txt in your browser. If you receive a 5xx error, it confirms that the issue is on the server side.
3. Contact Your Hosting Provider: If the file is not accessible, reach out to your hosting provider for assistance. They can help identify and resolve any server-related issues.
4. Review Server Logs: If you have access to server logs, check for any errors related to requests for the robots.txt file.

If you continue to experience issues, please reply to this ticket with your site URL so we can assist you further. A human agent from our support team will take over and provide more specific guidance.

Thank you for raising this concern, and we hope to resolve it quickly for you!

Hello,

Your robots.txt is working fine. You can also validate it using this tool:
https://rankmath.com/tools/robots-txt/

Regarding the error 500, they are usually caused due to server misconfiguration or other server issues.

You can share the screenshot with the response code or error log so we can check it as well. You can upload screenshots using this tool and add the link here.

Also, you should not use the Product schema on your WordPress Pages.

Please note that the Product Schema is only for the actual product and not for anything else. You can check this link for more information:
https://developers.google.com/search/docs/data-types/product#guidelines

Here’s a link for more information on why the ?srsltid= parameter is added to your URLs on the search results pages:
https://support.google.com/analytics/thread/288691031/why-google-started-to-use-srsltid-parameter-recently?hl=en

Looking forward to helping you.

Hello,

Since you can access the robots.txt file directly, but Google Search Console still reports a 5xx error, this could be due to temporary server issues, firewall restrictions, or bot-blocking settings by your hosting provider.

Please try the following:

– Check if any security plugins or server rules (like Cloudflare or mod_security) are blocking Googlebot.

– Temporarily disable any firewall or security settings and recheck in Google Search Console.

If the issue persists, we recommend reaching out to your hosting provider to ensure Googlebot isn’t being blocked at the server level.

2. Concerning the product schema you had applied on all pages, unfortunately, bulk editing options are only available to PRO users. You will need to edit them singly and remove the schema.

We hope this helps. Please let us know if you have further questions or concerns.

Thank you.

Hello,

Please share the affected page or post URL here so we can take a closer look.

Looking forward to helping you.

Hello,

The page doesn’t have schema even without the ?srsltid query parameter in the URL.

Since these are regular pages/posts, you can use the default Article schema.

Please note that the ?srsltid parameter is not coming from our plugin. To check how Google discovered such URLs, you can check them in the URL Inspection tool in your Google Search Console.

Don’t hesitate to get in touch with us if you have any other questions.

Hello,

The redirect on that page is WordPress redirecting the URL without a trailing slash to the version with one since that is the URL structure of your site.

You can use the URL inspection tool of Google search console to find out where Google is discovering that URL ( referring page) and remove/modify it to the version with a trailing slash.

We hope this helps. Please let us know if you have further questions or concerns.

Thank you.