URL returned a bad HTTP response code – Facebook Share Debugger

#1110617
  • Lancelot Lanceloty

    2 weeks, 3 days ago

    Rank Math free

    I am facing a strange error with Facebook debugger and my site: https://www.redball.gr/v2 and was hoping that with Rank Math i could solve it. Although i have set the share image and all settings, still show unresolved. Specifically i have this error: URL returned a bad HTTP response code. which shows more info as: This response code could be due to a robots.txt block. Please allowlist facebookexternalhit on your sites robots.txt config to utilize Facebook scraping

    Any thoughts on that?

Viewing 5 replies - 1 through 5 (of 5 total)
  • RankBot

    2 weeks, 3 days ago

    Hello,

    We apologize for the issue you are facing with the Facebook Share Debugger. It seems that the URL you provided is returning a bad HTTP response code, which can often be related to the settings in your robots.txt file.

    To resolve this issue, you need to ensure that the facebookexternalhit user agent is allowed in your robots.txt file. Here’s how you can do that:

    1. Edit Your robots.txt File: You can navigate to your robots.txt file in Rank Math by going to WordPress Dashboard → Rank Math SEO → General Settings → Edit robots.txt.

      Edit robots.txt in Rank Math

    2. Add Allow Rule: Add the following line to allow Facebook’s crawler:

      User-agent: facebookexternalhit
Allow: /

    3. Save Changes: After making the changes, make sure to save them.

    4. Check URL Accessibility: You can also check if your URL is accessible and returning a 200 status code using the HTTP Status Checker. Enter your URL and click Check Status to see if there are any issues.

      Check if URL is accessible

    5. Use Facebook Sharing Debugger: After making these changes, go back to the Facebook Sharing Debugger and scrape your URL again to see if the issue is resolved.

    If you continue to experience problems, please reply to this ticket with any additional details, and a human agent from our support team will assist you further. Thank you for raising this concern!

    Sources:
    No Information is Available for This Page » Rank Math
    How to Fix ‘Blocked by robots.txt’ Error in Google Search Console » Rank Math

    This is an automated reply from Rank Math Support Bot. Your ticket has been assigned to a human agent and we will get back to you shortly.

    Rakibuzzaman Siam

    2 weeks, 2 days ago

    Hello,

    We can see the Facebook Debugging tool is returning a 403 error for your website:

    In this case, you can get in touch with your hosting provider and make sure the Facebook bot is not blocked on their firewall.

    Also, your /robots.txt is currently returning a 404 error. However, it is working properly at /?robots=1. In this case, please try to clear your website cache, including any server-level cache, and check again.

    If the issue persists, please follow the steps mentioned in this guide and see if that works for you: https://rankmath.com/kb/fix-common-robots-txt-issues/#robots-txt-returns-blank-page-or-404-error

    Let us know how it goes. Looking forward to helping you.

    Thank you.

    Lancelot Lanceloty

    2 weeks ago

    Rank Math free

    Ty for your time devoted on this and the replies. I will try what u suggested and get back to you 🙂

    Lancelot Lanceloty

    1 week, 5 days ago

    Rank Math free

    Hello again,
    It seems i didn’t have a robots.txt file and also in General Settings of your plugin, robots.txt was showing that robots.txt file is not writable.

    I added a file within my root and add the code for fb you wrote above, and now the facebook debugger shows the following:
    Missing Properties
    The following required properties are missing: fb:app_id
    Also within my wordpress admin it still showing that robots.txt is not writable. How i can make your plugin scan again and find what i uploaded, so i can edit it through your plugin in the future?

    Jeremy

    1 week, 2 days ago

    Hello,

    Here’s a guide you can follow to fix that: https://rankmath.com/kb/cant-edit-robots-txt/

    Here’s also a guide you can follow to fix the missing fb:app_id: https://rankmath.com/kb/fb-app-id-warnings/

    Let us know how that goes. Looking forward to helping you with this one.

Viewing 5 replies - 1 through 5 (of 5 total)

You must be logged in to reply to this ticket.