Hello,
We apologize for the issues you are facing with your website’s robots.txt file and sitemap URL. Thank you for bringing this to our attention.
Addressing the Robots.txt File Not Writable Issue
The error indicating that the robots.txt file is not writable typically occurs when Rank Math cannot modify your robots.txt file. To resolve this, you can add the following code to your site:
/**
* Allow editing the robots.txt & htaccess data.
*
* @param bool Can edit the robots & htacess data.
*/
add_filter( 'rank_math/can_edit_file', '__return_true' );
Additionally, check your wp-config.php
file for the following line:
define (‘DISALLOW_FILE_EDIT’, true);
If it exists, change it to:
define ('DISALLOW_FILE_EDIT', false);
If it does not exist, you can add it. If you are unable to access your wp-config.php
file, please contact your hosting provider for assistance.
Fixing the 404 Error for the Sitemap URL
To address the 404 error for the sitemap URL, ensure that there are no conflicting plugins or physical files that might be causing this issue. You can check if another plugin is creating a sitemap or if a physical sitemap file exists in your site’s root directory. If so, you may need to disable that plugin or remove the physical file.
After making these changes, clear your website and server-level cache and test your sitemap URL again. You can refer to this guide on how to clear cache in WordPress.
Why Did Rank Math Automatically Create This?
Rank Math automatically creates a virtual robots.txt file to help manage SEO settings directly from the WordPress dashboard. This means you do not need a physical robots.txt file in your website’s root folder. If a physical file exists, it can conflict with Rank Math’s virtual file, leading to issues like the ones you’re experiencing.
Next Steps
Please reply to this ticket with your site URL so we can assist you further if needed. A human agent from our support team will take over and provide additional help if necessary.
We appreciate your patience as we work to resolve these issues for you!
Sources:
I Can’t Edit My Website’s Robots.txt » Rank Math
Common robots.txt Issues & How to Fix Them » Rank Math