How to write the robots. txt of zblog? Sample download of robots. txt file for zblog

Robots.txt is a very important thing for webmasters who compare seo. Frequently, friends ask zblog how to write robots? This article will explain in detail.

First, let's explain What is robots.txt ？ What does robots.txt do ？

Robots.txt is the first file to view when visiting a website in a search engine. When a search spider visits a site, it will first check whether robots.txt exists in the root directory of the site. If it does, the search robot will determine the scope of access according to the contents of the file; If the file does not exist, all search spiders will be able to access all pages on the website that are not protected by passwords.

The above content comes from Baidu Encyclopedia.

After reading the above content, we can understand that robots.txt tells the searchers which pages to include, so we need to know which folders in the zblog program are for spiders and which folders cannot be crawled by spiders. Please refer to“ https://www.txcstx.com/post/1085.html ”。

Let's give you a demonstration robots.txt, which is the robots.txt file of our website:

 User-agent: * Disallow: /zb_users/ Disallow: /zb_system/ Sitemap: http://www.txcstx.cn/sitemap.xml

The robots.txt above means that spiders should not climb the zb_users and zb_system folders. In addition, the "Sitemap" file on this site is generated by the "SEO Tools Collection" plug-in, and you can download other plug-ins（ https://app.zblogcn.com/search.php?q=Sitemap ）Generate Sitemap file.