Hello,
To remove the feeds and specific URLs from your sitemap, you can use the robots.txt file to block search engines from crawling these URLs. You can add the following rules to your robots.txt file:
User-Agent: *
Disallow: /post-1/feed/
Disallow: /post-2/feed/
Disallow: /post-2/?amp=1
Disallow: /post-2/?utm_source=rss&utm_medium=rss&utm_campaign=post-2
By adding these rules, you are instructing search engines not to crawl these specific URLs, effectively removing them from the sitemap.
If you have any further questions or need additional assistance, feel free to ask.
Sources:
Troubleshooting the “Couldn’t Fetch” Error in Google Search Console » Rank Math
How to Exclude Content From Sitemap » Rank Math
How to Remove Date Snippets from Search Results » Rank Math
Hello,
Thank you for contacting Rank Math support and we apologize for the inconvenience.
Rank Math does not include in its sitemap any pages that have been set to noindex, so you should be able to resolve this by setting the URLs to noindex. Here’s a code you can use:
add_filter( 'rank_math/frontend/robots', function( $robots ) {
// Get the current page URL
$current_url = home_url( $_SERVER['REQUEST_URI'] );
// Define an array of pages to change the robots tag to noindex
$pages = array(
'https://test.com/post-1/feed/',
'https://test.com/post-2/?amp=1'
);
// Check if the current page is in the array
if ( in_array( $current_url, $pages ) ) {
// Change the robots tag to noindex
$robots['index'] = 'noindex';
}
// Return the modified robots array
return $robots;
});
You can learn how to add this code to your website here: https://rankmath.com/kb/wordpress-hooks-actions-filters
We hope this helps you resolve the issue. If you have any other questions or concerns regarding Rank Math, please feel free to contact us again. We are always happy to help.
Thank you for choosing Rank Math!
Hi,
we never created these types of pages, can you tell how and why these types of pages are created and included in the sitemap?
Thanks
Kamal
Hello,
Feed URLs are only the basic code versions of your actual content pages and are generated by WordPress. If you want to disable them on your site, you can follow this guide:
https://kinsta.com/knowledgebase/wordpress-disable-rss-feed/
However, Rank Math should not include the feed and amp URLs to your sitemap. You can share your sitemap URL with us so that we can check the issue further for you.
Looking forward to helping you.
Thank you.
these urls are not in the sitemap but google search consloe show them
Hello,
Thank you for clarifying that these URLs are not included in your sitemap but are showing up in Google Search Console.
This situation can occur for a few reasons, and there are steps you can take to address it.
Although these URLs are not in your sitemap nor created by you, Google can still find and index them through other means, such as links from other pages or direct crawling.
Feed URLs are automatically generated by WordPress and are used for RSS feeds.
AMP (Accelerated Mobile Pages) URLs are created if you have an AMP plugin or feature enabled.
URLs with UTM parameters (like ?utm_source=rss) are typically generated by marketing or analytics tools for tracking purposes.
If you don’t use RSS feeds, you can disable them. Follow the guide provided by Rakibuzzaman Siam on disabling RSS feeds in WordPress: https://kinsta.com/knowledgebase/wordpress-disable-rss-feed/
If these URLs are already indexed by Google and you want them removed, you can use the URL Removal Tool in Google Search Console to temporarily block URLs from Google Search results.
We hope that helps, and please don’t hesitate to get in touch if you have any other questions.
Thank you.
Hello,
In addition to the previous suggestions, another effective method to prevent these URLs from being crawled and indexed by search engines is to use the robots.txt file.
You can add specific rules to this file to instruct search engines not to crawl certain URLs on your site.
You can add the following lines to your robots.txt file to prevent search engines from crawling the specific URLs:
User-agent: *
Disallow: /*/feed/
Disallow: /*?amp=
Disallow: /*?utm_source=
Here’s a guide on how to edit your robots.txt using Rank Math:
https://rankmath.com/kb/how-to-edit-robots-txt-with-rank-math/
We hope that helps, and please don’t hesitate to get in touch if you have any other questions.
Thank you.
Hello,
We are glad we could address your concern.
Please feel free to create a new forum topic if you do have another question in the future.
It will be our pleasure to assist you again.
Thank you.
