People who do website optimization know that although search engines can recognize and include dynamic links now, static links are relatively more friendly to search engines. So when setting link URLs for websites, especially children's shoes that use programs like WordPress to do independent blogs, they always like to set wordpress fixed connection pseudo static, but there is a problem that Baidu will still include your dynamic links. Let's talk about my situation below:
This is the problem I encountered in my Xiaoshan blog. Yesterday I posted an article. Today, I checked the collection and found that dynamic links and static links with pseudo static settings were included. SEO people know that it is not good when there are many different links pointing to the same page, so we should tell Baidu Spider not to crawl dynamic links.
Static links included by Baidu
Dynamic links included by Baidu
Many people say that setting 301 jump can solve the problem. I know that if there are many home page URLs, 301 jump is used. For example, this happens when both www and no www are parsed, so we use 301 jump to solve this problem.
But how to solve this problem when there are so many links on the article content page? I solved this problem through robots.txt. In robots.txt, Baidu spiders are prohibited from crawling dynamic links, so I added this sentence in robots.txt: Disallow:/** Access to all dynamic pages in the website is prohibited.
