Although Robots is a very basic thing on the website, many friends still don't know what it is used for or how to write it. Today, we will talk about what robots are and how to write robots on foreign trade websites
1. What is Robots
Robots is a protocol used among web crawlers (like GG spiders). Robots set which spiders can and cannot crawl which pages. It has no legal effect and is completely conscious. It is placed in the document robots. txt.
2. How to write website robots
Website inclusion and crawling prohibited
The first step in building a WP website is to write it, because your website has not been completed and is not suitable for inclusion
User-agent: *
Disallow: /
Website open inclusion and crawling
Generally speaking, there is no standard answer to the writing of robots on the website. Because it is an open content, the purpose is to tell what can be done and what can not be done, so it is entirely up to you to decide.
Recent articles of professional SEO plug-in YOAST SEO show that there is no need to restrict anything, which is equivalent to streaking.
Especially for AVADA themed websites, if there is any restriction on the Google webmaster tool background, they will be prompted to report errors, so they are open to the website for inclusion. The released robots are as follows.
User-agent: * Allow: /
3. Where to put robots files
After the robots.txt file is written, it can be directly uploaded to the root directory of the website, that is, the same path as wp admin, wp content, and wp includes.
The simple and quick way to scale is to use the editing function of the yoast seo plug-in. You can edit robots online, which is very convenient. Please refer to Use of yoast seo plug-in
4. If Robots prohibits it, will others not see my website?
No, robots are network protocols that work for web crawlers but not for people.
To prevent others from seeing your website - install the under construction plug-in and enable it to block your website
If the website does not include robots, write robots and put them under the root directory of your website
