Recently, a friend asked: I found that Google did not include your tag page on your site. What about mine? Yes or mostly? Advocate Meng thinks that many novices may not know the reason, so they just write an article.
1. Why Google doesn't include tag pages
In fact, Advocate Meng uses Robots.txt as a limited collection. For Robots.txt, see《 What is Robots.txt/What is it used for/How to write it 》。 You can also directly check the Robots.txt of the private plots: https://www.cmhello.com/robots.txt
be careful ：
1. I am not familiar with Robots.txt, so the following rules are not standardized, concise, or even wrong. If you see the wrong place, I hope you can give me some advice. Thank you very much.
2. Everyone's link style is different, Do not copy my Robots.txt, or you will be responsible for the consequences oh
As can be seen from my robots.txt, I prohibit all search engines from including tags, categories, comments, feeds and other page types, so search engines will not include these pages, and gradually remove the prohibited types of articles, as shown below
Obviously Baidu has basically taken all of my Tag and category page Yes, Google has not completely removed it yet (just put the tag and category page collection results on the last few pages) Be careful. You can just site my station.
2. Which page types are prohibited
To know which pages should be prohibited, it is recommended to use Google Webmaster Tools This is a very good tool. If you haven't used it yet, please use it quickly.
be careful : Advocate Mengban search engines from including tags and category pages. It is only an SEO test. Please do not follow it blindly, or you will bear all the consequences.
(1) WordPress usually requires prohibited pages, please refer to Wange How to write the boss: http://wange.im/robots.txt
(2) You can also add pages you don't want to include according to the rules. You can also use the Google Webmaster Tools [Grab Error] to view the [Not Found] and [Not Accessible] pages:
3. How to let search engines remove included articles
Write the [Not Found] pages in the above figure into robots.txt, prompting the search engine to remove the inclusion. Then, you can add the above [Not Found] URL in [Website Configuration]>[Grab Tool Permission]>[Delete URL]>and submit the deletion application, so that Google will process it.
Note: You can also see from the above figure that you can [Test robots. txt] and [Generate robots. txt]
Summary
Through robots.txt, it is very convenient to disable search engine inclusion, or to remove articles that have been included. But when writing robots.txt, you must pay attention to the details and remember to use Google Webmaster Tools Check whether robots.txt is correct and valid. That's all for today. If you don't understand, you can leave a message. I suggest you google more.
We strongly recommend you to watch this video: What operations of the stationmaster will lead to authority reduction and being killed
PS: If something is wrong in this article, I hope you can correct it in time; If you know more about robots. txt and Google Webmaster Tools Thank you for your contribution and sharing.