What does noindex mean?
When Google spider (crawler) visit a URL (page URL) and include that URL into search engine result page then we used to say that Google has indexed that URL.
So adding a page (URL) into search engine result page is known as ‘Indexing an URL’.
‘Noindex’ is just opposite to indexing process. When we don’t want a page or URL to be listed in search engine then we made it ‘noindex’.
When a Google spider (crawler) found a ‘noindex’ link then it is an indication for crawler that this page should not be included in search result page.
Why we need to set wordpress categories and tags as ‘noindex’?
There are sites (personal sites or sites with confidential data) that don’t want to expose its data so usually owner of these sites mark them as noindex.
Above scenario is accepted and understood but why wordpress categories and tags?
“We set our categories and tags as ‘noindex’ to avoid duplicate data or poor data penalty”.
Let me explain why and how
Suppose you have one category mobile in your blog. You write an article ‘Top 10 android mobiles’ and linked this article with ‘Mobile’ category and you added tag as ‘Android Mobiles’.
Now wordpress will show your article (or excerpt) with following URLs
URL of your article – http://www.xyz.com/12042013/top-10-android-mobiles
URL of your Category – http://www.xyz.com/Category/Mobiles
URL of your Tag – http://www.xyz.com/tag/android-mobiles
URL of you home page – http://www.xyz.com
If you see technically all the four URLs pointing to same content and that give an impression of duplicate content.
So it is a standard practice that we should set our wordpress ‘categories’ and ‘tags’ as ‘noindex’ so that crawler will not index them into search engine result page.
Now you all are thinking that we have set categories and tag as noindex but still two URL pointing to same data (Page URL and Home Page URL).
Google has mentioned that its algorithm is intelligent enough to distinguish between a home page (domain URL) and actual page URL so we need not to worry about it.
I hope till now you got enough idea why we need to make our category and tag noindex.
How to make wordpress blog categories and tags noindex?
There are several ways to do it
Using robot file
This is the easiest way to set your categories and tags as noindex. Just add below lines in your Robots.txt file (you will find it in your wordpress blog folder in your hosting).
Above lines will stop crawler to crawl your categories and tags.
All in one SEO plugin- If you go to setting page of your AIO-SEO plugin then you will find option to make your categories and tags noindex.
WordPress Meta Robot- This is another great plugin with easy to use interface. It allows you set your page, post, categories, tags as ‘noindex’.
How to remove indexed Categories and tags from search engine?
What to do if your categories and tags are already indexed in search result page. Once you made above changes and create your updated sitemap and submit it to Google (using Webmaster Tool) then Google will de-indexed your categories and tags after some time automatically.
Or alternatively you can use ‘Remove URL’ functionality provided by ‘Google Webmaster Tool’.