403 status code preventing SEO analysis

#188977
  • Resolved Christopher Morgan
    Rank Math free

    I’m trying to perform the SEO Analysis but keep getting the “API Error: HTTP 400 error.” error.

    I checked with the https://httpstatus.io/ website and it seems that my site is throwing a 403 status code BUT is still loading perfectly fine:

    https://BounceLearningKids.com

    I enabled maintenance mode and disabled everything but still getting the same above error.

    How can it be that all the web pages are loading perfectly find yet I’m still getting the above error?

Viewing 14 replies - 1 through 14 (of 14 total)
  • Prabhat
    Rank Math agency

    Hello,

    Thanks for contacting us and we regret the inconvenience caused.

    I tested your website using https://httpstatus.io/ but I was unable to replicate the issue. Your website gives a 200 (OK) status. Please have a look at this screenshot for a quick reference: https://i.rankmath.com/eWfh6L

    Regarding the error in the SEO Analysis, it appears your web server is blocking access to our API. Please make sure to whitelist the IP 134.90.148.114 and the user-agent RankMathAPI/2.0.0; https://rankmath.com.

    If you are not sure, please ask your web hosting provider to do it for you.

    Once done, you can run the SEO Analysis once again.

    Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.

    Run the check again. The first time I ran the check, it also showed a 200 status code but when I run it again after that, it consistently throws a 403. I’ve replicated this with multiple browsers and I get the same behavior each time.

    I’ve already whitelisted the IP address (in both the WHM CSF rules as well as in Wordfence. I’m not sure how to whitelist a user agent, as I’ve never done that before. Can you point me in the right direction?

    Nishi Kant
    Rank Math business

    Hello,

    Thank you for sharing further details.

    You are requested to update the user agent access at your hosting server, and please check if you need to whitelist it in any of the security plugins.

    We hope that helps. Please let us know if you need help with anything else.

    Thanks

    Will that make a difference given the 403 status code?

    Hello,

    Thank you for the follow-up.

    Whitelisting the IP and the user-agent RankMathAPI/2.0.0; https://rankmath.com will only fix the SEO Analysis issue.

    In this case, you may need to get in touch with your web host since httpstatus is also reporting 403 for your website’s status.

    I hope that helps. Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.

    I run my own VPS running cPanel WHM. I’ve no idea how to whitelist the user agent and can’t find anything on the internet that explains how to do this. Can you point me in the right direction?

    I’m starting to suspect the website that you recommended to check the status code. With an alternative status checker, I do get a consistent 200 returned:

    https://smartranking.nl/tools/http-status-checker/

    Results:

    HTTP/2 200
    date: Tue, 27 Apr 2021 20:04:17 GMT
    content-type: text/html; charset=UTF-8
    set-cookie: __cfduid=dbb98f717e90ccb6b9c884327cdc2e9a81619553856; expires=Thu, 27-May-21 20:04:16 GMT; path=/; domain=.bouncelearningkids.com; HttpOnly; SameSite=Lax
    cf-edge-cache: cache,platform=wordpress
    link: ; rel=”https://api.w.org/”, ; rel=”alternate”; type=”application/json”, ; rel=shortlink
    set-cookie: apbct_timestamp=1619553856; path=/; HttpOnly; SameSite=Lax
    set-cookie: apbct_site_landing_ts=1619553856; path=/; HttpOnly; SameSite=Lax
    set-cookie: apbct_page_hits=1; path=/; HttpOnly; SameSite=Lax
    set-cookie: apbct_cookies_test=%257B%2522cookies_names%2522%253A%255B%2522apbct_timestamp%2522%252C%2522apbct_site_landing_ts%2522%252C%2522apbct_page_hits%2522%255D%252C%2522check_value%2522%253A%2522deba27dd0f0c574ddb765c765215b5bf%2522%257D; path=/; HttpOnly; SameSite=Lax
    cache-control: max-age=2592000
    expires: Thu, 27 May 2021 20:04:15 GMT
    vary: User-Agent
    referrer-policy: no-referrer-when-downgrade
    cf-cache-status: DYNAMIC
    cf-request-id: 09b687c235000002d2fa2fc000000001
    expect-ct: max-age=604800, report-uri=”https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct”
    report-to: {“endpoints”:[{“url”:”https:\/\/a.nel.cloudflare.com\/report?s=FceUhuTKjTf%2FmPJwy2W0rs%2FtprEov0ZMjIfpWVdGxIuyLZyjLP7jQVqqIkc2JiYIkX%2Bx%2FQce1CXazVbXYq3XDMCuKjopy3i0otI9NdeiaXZO4cD2CchJ”}],”max_age”:604800,”group”:”cf-nel”}
    nel: {“max_age”:604800,”report_to”:”cf-nel”}
    server: cloudflare
    cf-ray: 646aa8b05ed402d2-MIA
    alt-svc: h3-27=”:443″; ma=86400, h3-28=”:443″; ma=86400, h3-29=”:443″; ma=86400

    I’ve since been in touch with the people behind httpstatus.io and they tell me that Cloudflare is responsible for the 403 being returned for their service. In actual fact, my website is presenting a normal 200 status code otherwise, so that cannot be the problem here.

    I’ve also since added 2 firewall rules in my Cloudflare configuration. One for the user-agent:

    (http.user_agent eq “RankMathAPI/2.0.0; https://rankmath.com”)

    …and one for the IP address

    (ip.src eq 134.90.148.114)

    Unfortunately, I’m still seeing the “API Error: HTTP 400 error.” error.

    I’ve since been in touch with the people behind httpstatus.io and they tell me that Cloudflare is responsible for the 403 being returned for their service. In actual fact, my website is presenting a normal 200 status code otherwise, so that cannot be the problem here.

    I’ve also since added 2 ‘allow’ firewall rules in my Cloudflare configuration. One for the user-agent:

    (http.user_agent eq “RankMathAPI/2.0.0; https://rankmath.com”)

    …and one for the IP address

    (ip.src eq 134.90.148.114)

    Unfortunately, I’m still seeing the “API Error: HTTP 400 error.” error.

    Hello,

    Thank you for the update.

    Please head over to Rank Math > Status & Tools > Database Tools > and click the Remove transients button
    Clear SEO Analysis

    Once done, clear your website’s cache and Start again the sitewide analysis in Rank Math > SEO Analysis.

    If the issue still persists, we might need to take a closer look at the settings. Please edit the first post on this ticket and include your WordPress & FTP logins in the designated Sensitive Data section.
    Sensitive Data Section

    It is completely secure and only our support staff has access to that section. If you want, you can use the below plugin to generate a temporary login URL to your website and share that with us instead:

    https://wordpress.org/plugins/temporary-login-without-password/

    You can use the above plugin in conjunction with the WP Security Audit Log to monitor what changes our staff might make on your website (if any):

    https://wordpress.org/plugins/wp-security-audit-log/

    I hope that helps. Thank you, and looking forward to your update.

    Hello,

    I have updated the sensitive data as requested. Can you please check further?

    Thank you.

    Hello,

    Thank you for the update, and apologies for the delay.

    Upon further investigation, it seems that our SEO Analysis is seeing anti crawler protection page which is generated from your Anti-Spam by CleanTalk active plugin
    Antri crawler

    I’ve tried to disable that plugin temporarily, and the SEO Analysis works fine.

    I hope that helps. Thank you, and please don’t hesitate to contact us anytime if you need further assistance with anything else.

    Awesome! Thank you for your persistence. It might be an idea for you to update your documentation to forewarn other users of the issues we encountered in this ticket:

    1. When using Cloudflare, the httpdstatus.io checks may result in a 403, giving the incorrect impression there is a problem with the website when that isn’t the case.

    2. The Rankmath SEO analysis tool won’t work when Cleantalk anti-spam protection is enabled.

    Thank you for all your help 🙂

    Hello,

    Thank you for the update, and we’re glad that this issue is already resolved.

    We appreciate your suggestions and forwarded them to our dev team.

    If you have any other concerns, please don’t hesitate to contact us anytime by creating a new ticket to assist you further with anything else.

    Thank you.

Viewing 14 replies - 1 through 14 (of 14 total)

The ticket ‘403 status code preventing SEO analysis’ is closed to new replies.