Recently used

Popular recommendation

Limit directory:

Each path must be preceded by: "/"

(leave blank as none) Sitemap:

Google is in xml format, Baidu is in html format

Retrieval interval:
All search engines:
  • ##
    allow
    refuse
Domestic search engines
  • Baidu
    default
    allow
    refuse
  • SOSO
    default
    allow
    refuse
  • Sogou
    default
    allow
    refuse
  • Youdao
    default
    allow
    refuse
Foreign search engines
  • Google
    default
    allow
    refuse
  • Bing
    default
    allow
    refuse
  • Yahoo
    default
    allow
    refuse
  • Ask/Teoma
    default
    allow
    refuse
  • Alexa/Wayback
    default
    allow
    refuse
  • Cuil
    default
    allow
    refuse
  • MSN Search
    default
    allow
    refuse
  • Scrub The Web
    default
    allow
    refuse
  • DMOZ
    default
    allow
    refuse
  • GigaBlast
    default
    allow
    refuse
Special search engine (robot)
  • Google Image
    default
    allow
    refuse
  • Google Mobile
    default
    allow
    refuse
  • Yahoo MM
    default
    allow
    refuse
  • Yahoo Blogs
    default
    allow
    refuse
  • MSN PicSearch
    default
    allow
    refuse

Please save the following results to Notepad, named robots.txt, and upload them to the root directory of the website

Tool Introduction

Robots.txt is a plain text file stored in the root directory of the site. Although its setting is simple, it is powerful. It can specify that search engine spiders only crawl the specified content, or prohibit search engine spiders from crawling part or all of the website content.

About the webmaster's home | contact us | Advertising services | Friendly Links | Website Dynamics | Copyright Notice | Talent recruitment | help

© CopyRight 2002-2024, CHINAZ.COM, Inc.All Rights Reserved. Min ICP Bei No. 08105208 Value added telecommunication business license Min B2-20120007 Yisu Cloud provides ECS support