|
Spider |
Hits |
% of Total |
Visits |
1 |
Bing Spider |
6,370,734 |
26.26% |
6,315 |
2 |
BaiDu Spider |
6,068,923 |
25.02% |
215,408 |
3 |
Google Spider |
3,511,346 |
14.47% |
89,406 |
4 |
Yandex Spider |
2,289,505 |
9.44% |
11,875 |
5 |
Yahoo Spider |
1,754,850 |
7.23% |
5,211 |
6 |
magpie-crawler/1.1+(U;+Linux+amd64;+en-GB ;++http:/ / www.brandwatch.net) |
1,086,651 |
4.48% |
114 |
7 |
Treato-Bot/1.0+(TREATO+LTD,+www.treato.co m,+mailto:+botadmin@treato.com) |
468,303 |
1.93% |
11 |
8 |
Wespe.de Spider |
372,593 |
1.54% |
210,753 |
9 |
Sogou Spider |
321,694 |
1.33% |
2,086 |
10 |
Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10 _10_1)+AppleWebKit/ 600.2.5+(KHTML,+like+Gecko)+Version/ 8.0.2+Safari/ 600.2.5+(Applebot/ 0.1;++http:/ / www.apple.com/ go/ applebot) |
253,188 |
1.04% |
13,335 |
11 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.4.5;+http:/ / www.majestic12.co.uk/ bot.php? +) |
221,797 |
0.91% |
1,893 |
12 |
Vegi+bot+(we+follow+your+robots.txt+setti ngs+before+crawling,+you+can+slow+down+th e+bot+by+change+the+Crawl-Delay+parameter +in+the+settings.if+you+have+an+enquiry,+ please+email+to:+abuse-report@terrykylese oagency.com) |
221,531 |
0.91% |
10,631 |
13 |
Mozilla/5.0+(Windows;+U;+Windows+NT+6.0;+ en-GB;+rv:1.0;+trendictionbot0.5.0;+trend iction+search;+http:/ / www.trendiction.de/ bot;+please+let+us+know+of+any+problems;+ web+at+trendiction.com)+Gecko/ 20071127+Firefox/ 3.0.0.11 |
181,844 |
0.75% |
120,357 |
14 |
Mozilla/5.0+(compatible;+BLEXBot/ 1.0;++http:/ / webmeup-crawler.com/ ) |
160,800 |
0.66% |
928 |
15 |
Google image crawler |
135,763 |
0.56% |
638 |
16 |
Mozilla/5.0+(compatible;+proximic;++http: / / www.proximic.com/ info/ spider.php) |
77,013 |
0.32% |
64,361 |
17 |
Toweyabot:+toweya.com |
71,090 |
0.29% |
537 |
18 |
Facebook external hit |
67,373 |
0.28% |
49,616 |
19 |
Mozilla/5.0+(compatible;+DotBot/ 1.1;+http:/ / www.opensiteexplorer.org/ dotbot,+help@moz.com) |
66,033 |
0.27% |
2,094 |
20 |
Clickagy+Intelligence+Bot+v2 |
44,306 |
0.18% |
27,218 |
21 |
robots |
39,643 |
0.16% |
2,551 |
22 |
Yandex Images Spider |
38,792 |
0.16% |
2,899 |
23 |
Mozilla/5.0+(compatible;+MegaIndex.ru/ 2.0;++http:/ / megaindex.com/ crawler) |
35,990 |
0.15% |
4 |
24 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.4.6;+http:/ / mj12bot.com/ ) |
35,894 |
0.15% |
386 |
25 |
MSN Spider |
31,393 |
0.13% |
143 |
26 |
Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10 _10_1)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 40.0.2214.111+Safari/ 537.36+moatbot |
28,739 |
0.12% |
9,411 |
27 |
Mozilla/5.0+(compatible;+GrapeshotCrawler / 2.0;++http:/ / www.grapeshot.co.uk/ crawler.php) |
27,936 |
0.12% |
9,674 |
28 |
Mozilla/5.0+(compatible;+SeznamBot/ 3.2;++http:/ / napoveda.seznam.cz/ en/ seznambot-intro/ ) |
23,653 |
0.10% |
1,019 |
29 |
BoardReader/1.0+(http:/ / spider.boardreader.com) |
22,837 |
0.09% |
4,249 |
30 |
Mozilla/5.0+(compatible;+Steeler/ 3.5;+http:/ / www.tkl.iis.u-tokyo.ac.jp/ ~crawler/ ) |
21,083 |
0.09% |
198 |
31 |
CCBot/2.0+(http://commoncrawl.org/ faq/ ) |
20,399 |
0.08% |
217 |
32 |
Mozilla/5.0+(compatible;+XoviBot/ 2.0;++http:/ / www.xovibot.net/ ) |
17,365 |
0.07% |
3,043 |
33 |
Mozilla/5.0+(compatible;+Linux+x86_64;+Ma il.RU_Bot/ 2.0;++http:/ / go.mail.ru/ help/ robots) |
15,211 |
0.06% |
1,745 |
34 |
MaxPointCrawler/Nutch-1.10+(maxpoint.craw ler+at+maxpointinteractive+dot+com) |
14,641 |
0.06% |
595 |
35 |
yacybot+(/global;+amd64+Linux+3.16.0-4-am d64;+java+1.8.0_40;+Europe/ en)+http:/ / yacy.net/ bot.html |
14,490 |
0.06% |
422 |
36 |
Mozilla/5.0+(compatible;+SemrushBot/ 1.1~bl;++http:/ / www.semrush.com/ bot.html) |
12,223 |
0.05% |
646 |
37 |
istellabot/t.1 |
10,422 |
0.04% |
176 |
38 |
YisouSpider |
9,014 |
0.04% |
1,386 |
39 |
Mozilla/5.0+(compatible;+MSIE+9.0;+Window s+NT+6.1;+Trident/ 5.0);+360Spider(compatible;+HaosouSpider; +http:/ / www.haosou.com/ help/ help_3_2.html) |
7,762 |
0.03% |
6,330 |
40 |
R6_FeedFetcher_(www.radian6.com/ crawler) |
7,697 |
0.03% |
131 |
41 |
Alexa Spider |
7,058 |
0.03% |
1,008 |
42 |
Mozilla/5.0+(compatible;+linkdexbot/ 2.0;++http:/ / www.linkdex.com/ bots/ ) |
5,273 |
0.02% |
2,149 |
43 |
Google Feedfetcher |
4,634 |
0.02% |
1,756 |
44 |
NutchExperiment/5.0+(compatible;+Data2Lif eBot/ 1.0;+http:/ / data2life.com;+support+at+data2life+dot+c om) |
4,580 |
0.02% |
146 |
45 |
Mozilla/5.0+(compatible;+Cliqzbot/ 1.0++http:/ / cliqz.com/ company/ cliqzbot) |
4,424 |
0.02% |
890 |
46 |
GetIntent+Crawler+(http:/ / getintent.com/ bot.html) |
2,962 |
0.01% |
1,472 |
47 |
Mediatoolkitbot+(complaints@mediatoolkit. com) |
2,396 |
0.01% |
168 |
48 |
Domain+Re-Animator+Bot+(http:/ / domainreanimator.com)+-+support@domainrea nimator.com |
2,298 |
0.01% |
6 |
49 |
Mozilla/5.0+(compatible;+coccocbot-image/ 1.0;++http:/ / help.coccoc.com/ searchengine) |
2,027 |
0.01% |
21 |
50 |
Mozilla/5.0+(compatible;+spbot/ 5.0.3;++http:/ / OpenLinkProfiler.org/ bot+) |
2,008 |
0.01% |
3 |
|
Subtotal |
24,218,181 |
99.83% |
885,631 |
|
Total |
24,258,883 |
100.00% |
898,166 |
|
