home page > Programmer development tools > Complete list of search engine spider IP addresses

Complete list of search engine spider IP addresses

proposal Idc Add the machine room firewall white list
Updated: 2023-12-5
Communication QQ group: six hundred and ninety-nine million six hundred and eleven thousand two hundred and eighty

Search Engines ASN data IP segment IP Home Spider & Robot UserAgent Relevant information
Baidu baidu.com AS4808
AS4808
AS4808
AS55967
AS23724
AS23724
AS23724
AS4134
AS4837
AS4837
AS137702
61.135.0.0/16
123.125.0.0/16
111.206.0.0/16
180.76.0.0/20
180.149.128.0/19
220.181.0.0/16
36.110.128.0/17
113.24.0.0/14
124.164.0.0/14
116.179.0.0/16
180.97.0.0/18
Beijing Unicom
Beijing Unicom
Beijing Unicom
Hong Kong Baidu Cloud
Beijing Telecom
Beijing Telecom
Beijing Telecom
Shanxi Yangquan Telecom
Shanxi Yangquan Unicom
Shanxi Yangquan Unicom
Jiangsu Nanjing Telecom Baidu Netcom
Baiduspider 61.135.186.13
123.125.71.138
111.206.221.38
180.76.15.138
180.149.133.38
220.181.32.38
36.110.199.138
113.24.225.138
124.166.232.138
116.179.32.138
180.97.35.138
Mozilla/5.0 (compatible; Baiduspider/2.0; + http://www.baidu.com/search/spider.html )
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko) Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0; + http://www.baidu.com/search/spider.html )
Mozilla/5.0 (compatible; Baiduspider-render/2.0; + http://www.baidu.com/search/spider.html )
Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0; + http://www.baidu.com/search/spider.html )
Mozilla/5.0 (iPhone;CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko)Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0;Smartapp; + http://www.baidu.com/search/spider.html )
Baiduspider-image+(+ http://www.baidu.com/search/spider.htm )
Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0
Mozilla/5.0 (Linux; Android 6.0.1; SOV33 Build/35.0.D.0.326) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.91 Mobile Safari/537.36
Mozilla/5.0 (Linux; *; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/76.0.3809.89 Mobile Safari/537.36 T7/11.18 SP-engine/2.14.0 baiduboxapp/11.18.0.12 (Baidu; P1 10) NABar/1.1
Mozilla/5.0 (Linux; *; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/76.0.3809.89 Safari/537.36 T7/11.18 SP-engine/2.14.0 baiduboxapp/11.18.0.12 (Baidu; P1 4.4.4) NABar/2.0
Mozilla/5.0 (Linux; *; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/76.0.3809.89 Mobile Safari/537.36 T7/11.18 SP-engine/2.14.0 baiduboxapp/11.18.0.12 (Baidu; P1 5.0.2) NABar/2.1
Mozilla/5.0 (Linux; *; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/76.0.3809.89 Mobile Safari/537.36 T7/11.18 SP-engine/2.14.0 baiduboxapp/11.18.0.12 (Baidu; P1 5.1)
Mozilla/5.0 (Linux; *; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/76.0.3809.89 Mobile Safari/537.36 T7/11.18 light/1.0 SP-engine/2.14.0 baiduboxapp/11.18.0.12 (Baidu; P1 6.0)
Mozilla/5.0 (Linux; *; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/76.0.3809.89 Mobile Safari/537.36 T7/11.18 SP-engine/2.14.0 baiduboxapp/11.18.0.12 (Baidu; P1 7.0)
Mozilla/5.0 (Linux; *; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/76.0.3809.89 Mobile Safari/537.36 T7/11.18 SP-engine/2.14.0 baiduboxapp/11.18.0.12 (Baidu; P1 7.1.1)
......
Sogou sogou.com AS18257
AS4134
AS4808
AS4808
AS4808
AS4808
AS23724
AS23724
AS23724
AS23724
AS23724
AS23724
AS17623
AS4816
AS4816
AS23724
118.184.176.0/22
123.180.0.0/14
123.125.0.0/16
61.135.0.0/16
123.126.0.0/16
111.202.0.0/16
36.110.128.0/19
220.181.0.0/16
106.120.128.0/17
49.7.0.0/16
218.30.96.0/19
106.38.0.0/16
58.250.0.0/16
183.36.96.0/19
43.231.96.0/22
49.7.64.0/18
Jiangsu Changzhou Telecom&Unicom
Hebei Zhangjiakou Telecom
Beijing Unicom
Beijing Unicom
Beijing Unicom
Beijing Unicom
Beijing Telecom
Beijing Telecom
Beijing Telecom
Beijing Telecom
Beijing Telecom
Beijing Telecom
Shenzhen Unicom, Guangdong Province
Shenzhen Telecom, Guangdong Province
China Guangdong Shenzhen Telecom&Unicom&Mobile
Beijing Telecom
Sogou web spider 118.184.177.138
123.183.224.138
123.125.125.181
61.135.189.138
123.126.113.138
111.202.100.138
36.110.147.70
220.181.125.138
106.120.173.138
49.7.20.138
218.30.103.50
106.38.241.188
58.250.125.37
183.36.114.102
43.231.99.68
49.7.116.138
Sogou web spider/4.0(+ http://www.sogou.com/docs/help/webmasters.htm#07 )
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25(compatible; Sogou web spider/4.0; + http://www.sogou.com/docs/help/webmasters.htm#07 )
360 so.com AS4812
AS4812
AS4837
180.153.0.0/16
180.163.128.0/17
42.224.0.0/12
Shanghai Telecom
Shanghai Telecom
Henan Luoyang Unicom
360Spider https://www.so.com/help/spider_ip.html
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36; 360Spider
Mozilla/5.0 (Linux; U; Android 4.0.2; en-us; Galaxy Nexus Build/ICL53F) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30; 360Spider
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
Shenma sm.cn AS37963
AS37963
AS37963
42.156.128.0/17
42.120.128.0/17
106.11.144.0/20
Alibaba Cloud, Hangzhou, Zhejiang
Alibaba Cloud, Hangzhou, Zhejiang
Alibaba Cloud, Hangzhou, Zhejiang
YisouSpider 42.156.138.138
42.120.236.138
106.11.159.138
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e YisouSpider/5.0 Safari/602.1
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.81 YisouSpider/5.0 Safari/537.36
YisouSpider
Bing. com AS59067
AS59067
AS59067
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
AS8075
202.89.232.0/21
103.255.140.0/22
103.25.156.0/24
111.221.16.0/20
157.56.0.0/16
199.30.16.0/20
65.52.0.0/14
13.64.0.0/11
157.55.0.0/16
207.46.0.0/16
40.64.0.0/10
40.76.0.0/14
40.80.0.0/12
104.40.0.0/13
131.253.24.0/21
131.253.32.0/20
191.232.0.0/13
20.40.0.0/13
23.103.64.0/18
64.4.0.0/18
52.160.0.0/11
Beijing Microsoft Cloud
Beijing Microsoft Cloud
Beijing Microsoft Cloud
Hong Kong Microsoft Cloud
Quincy, Washington, USA
Boyton, VA, USA
Quincy, Washington, USA
Santa Clara, California, USA
Quincy, Washington, USA
San Antonio, Texas, USA
Virginia, USA
Boyton, VA, USA
Amsterdam, Netherlands
Cheyenne, Wyoming, USA
Boyton, VA, USA
Microsoft Cloud, Boyton, Virginia, USA
Microsoft Cloud, Campinas, Sao Paulo, Brazil
Microsoft Cloud, Chennai, Tamil Nadu, India
Microsoft Cloud in Quincy, Washington, USA
Microsoft Cloud Dublin, Dublin, Ireland
Santa Clara, California, USA
bingbot 202.89.235.138
103.255.141.138
103.25.156.138
111.221.28.138
157.56.0.138
199.30.20.138
65.55.210.138
13.66.139.138
157.55.39.138
207.46.13.138
40.77.167.138
40.77.168.138
40.90.159.138
104.47.224.138
131.253.24.138
131.253.38.138
191.232.136.86
20.43.120.28
23.103.64.138
64.4.22.138
52.160.?.?
https://www.bing.com/toolbox/verify-bingbot-verdict
Mozilla/5.0 (compatible; bingbot/2.0; + http://www.bing.com/bingbot.htm )
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 BingPreview/1.0b
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; + http://www.bing.com/bingbot.htm )
Google google.com AS24424
AS15169
AS15169
AS15169
AS15169
AS15169
AS15169
AS3561
203.208.32.0/19
192.178.0.0/15
74.125.0.0/16
72.14.192.0/18
64.233.160.0/19
66.249.64.0/19
66.102.0.0/20
64.68.64.0/19
Beijing Telecom
Changhua County, Taiwan Province, China
Monkscona, South Carolina, USA
Darth, Oregon, USA
Consell Bluffs, Iowa, USA
Monkscona, South Carolina, USA
Google Inc
California, USA
Googlebot 203.208.60.138
192.178.5.138
74.125.210.48
72.14.199.224
64.233.172.36
66.249.64.138
66.102.6.138
64.68.90.138
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.92 Mobile Safari/537.36 (compatible; Googlebot/2.1; + http://www.google.com/bot.html )
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; + http://www.google.com/bot.html )
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; + http://www.google.com/bot.html ) Safari/537.36
Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 5 Build/JOP40D) AppleWebKit/535.19 (KHTML, like Gecko; googleweblight) Chrome/38.0.1025.166 Mobile Safari/535.19
Mozilla/5.0 (compatible; Googlebot/2.1; + http://www.google.com/bot.html )
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.74 Mobile Safari/537.36 (compatible; Googlebot/2.1; + http://www.google.com/bot.html )
Googlebot/2.1 (+ http://www.google.com/bot.html )
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36 (compatible; Google-Read-Aloud; + https://support.google.com/webmasters/answer/1061943 )
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Mobile Safari/537.36 (compatible; Googlebot/2.1; + http://www.google.com/bot.html )
Googlebot-Image/1.0
Googlebot-Video/1.0
Today's headline toutiao.com AS4134
AS4808/AS9808/AS23724
AS4837
AS4837
111.224.0.0/14
220.243.128.0/18
110.240.0.0/12
60.8.0.0/15
Hebei Zhangjiakou Telecom
Beijing Unicom BGP
Hebei Zhangjiakou Unicom
Hebei Zhangjiakou Unicom
Bytespider 111.225.149.138
220.243.135.138
110.249.201.138
60.8.151.138
Mozilla/5.0 (compatible; Bytespider;[ https://zhanzhang.toutiao.com/ ] AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[70.0.0.0]( http://70.0.0.0 ) Safari/537.36
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; [ https://zhanzhang.toutiao.com/ ]
Mozilla/5.0 (iPhone; CPU iPhone OS 7\_1\_2 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Version/7.0 Mobile Safari/537.36 (compatible; Bytespider; [ https://zhanzhang.toutiao.com/ ]
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.3072.1769 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2125.1549 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.7282.1240 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2466.1117 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.7004.1867 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.5219.1367 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.1666.1244 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2711.1431 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.7862.1276 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.8502.1519 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.1865.1898 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.1953.1175 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.3062.1821 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.9548.1423 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.5095.1942 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.9321.1429 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.8793.1741 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.1431.1066 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.5735.1944 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.6835.1927 Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
Mozilla/5.0 (*) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/* Mobile Safari/537.36; Bytespider; https://zhanzhang.toutiao.com/
......
unknown AS4808 220.249.0.0/18 Beijing Unicom Suspected Sogou web spider 220.249.46.120
unknown AS4847 106.39.0.0/16 Beijing Telecom Suspected Sogou web spider 106.39.246.138
unknown AS4808 211.94.64.0/18 Beijing Unicom Suspected 211.94.109.138
unknown AS4808/AS9808/AS23724 114.118.0.0/21 Beijing BGP Suspected 114.118.6.208
unknown AS4837 60.10.0.0/16 Hebei Langfang Unicom Suspected 60.10.84.100
unknown AS9808 223.96.0.0/12 Hebei Baoding Mobile Suspected 223.104.103.83
unknown AS4835 113.142.0.0/17 Xi'an Telecom, Shaanxi Province Suspected 113.142.72.118