|
Spider |
Hits |
% of Total |
Visits |
1 |
Google Spider |
11,394,964 |
44.36% |
53,401 |
2 |
Bing Spider |
10,237,684 |
39.85% |
7,094 |
3 |
Wespe.de Spider |
713,808 |
2.78% |
5,136 |
4 |
BaiDu Spider |
592,901 |
2.31% |
122,087 |
5 |
Yandex Spider |
521,242 |
2.03% |
51 |
6 |
Yahoo Spider |
363,173 |
1.41% |
6,273 |
7 |
Mozilla/5.0+(compatible;+archive.org_bot+ +http:/ / www.archive.org/ details/ archive.org_bot) |
306,596 |
1.19% |
364 |
8 |
Mozilla/5.0+(iPhone;+U;+CPU+iPhone+OS+4_1 +like+Mac+OS+X;+en-us)+AppleWebKit/ 532.9+(KHTML,+like+Gecko)+Version/ 4.0.5+Mobile/ 8B117+Safari/ 6531.22.7+(compatible;+Googlebot-Mobile/ 2.1;++http:/ / www.google.com/ bot.html) |
203,294 |
0.79% |
226 |
9 |
Mozilla/5.0+(compatible;+BLEXBot/ 1.0;++http:/ / webmeup-crawler.com/ ) |
196,928 |
0.77% |
35 |
10 |
MSN Spider |
178,111 |
0.69% |
54,228 |
11 |
Facebook external hit |
157,166 |
0.61% |
43,871 |
12 |
Mozilla/5.0+(iPhone;+CPU+iPhone+OS+6_0+li ke+Mac+OS+X)+AppleWebKit/ 536.26+(KHTML,+like+Gecko)+Version/ 6.0+Mobile/ 10A5376e+Safari/ 8536.25+(compatible;+Googlebot-Mobile/ 2.1;++http:/ / www.google.com/ bot.html) |
154,970 |
0.60% |
140 |
13 |
Mozilla/5.0+(compatible;+Linux+x86_64;+Ma il.RU_Bot/ 2.0;++http:/ / go.mail.ru/ help/ robots) |
115,312 |
0.45% |
2,617 |
14 |
Yandex Images Spider |
99,937 |
0.39% |
8 |
15 |
Google image crawler |
70,549 |
0.27% |
238 |
16 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.4.4;+http:/ / www.majestic12.co.uk/ bot.php? +) |
62,482 |
0.24% |
1,000 |
17 |
Mozilla/5.0+(compatible;+Ezooms/ 1.0;+ezooms.bot@gmail.com) |
32,257 |
0.13% |
2,630 |
18 |
WeSEE:Search/0.1+(Alpha,+http:/ / www.wesee.com/ en/ support/ bot/ ) |
22,978 |
0.09% |
1,529 |
19 |
Mozilla/5.0+(compatible;+proximic;++http: / / www.proximic.com/ info/ spider.php) |
18,930 |
0.07% |
6,919 |
20 |
BoardReader/1.0+(http:/ / spider.boardreader.com) |
18,483 |
0.07% |
34 |
21 |
Sogou Spider |
15,703 |
0.06% |
609 |
22 |
Mozilla/5.0+(compatible;+SemrushBot/ 0.97;++http:/ / www.semrush.com/ bot.html) |
14,311 |
0.06% |
61 |
23 |
Mozilla/5.0+(compatible;+EasouSpider;++ht tp:/ / www.easou.com/ search/ spider.html) |
14,024 |
0.05% |
1,430 |
24 |
Mozilla/5.0+(compatible;+special_archiver / 3.1.1++http:/ / www.archive.org/ details/ archive.org_bot) |
11,931 |
0.05% |
679 |
25 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ zh-CN;+rv:1.8.0.11)++Firefox/ 1.5.0.11;+360Spider |
11,580 |
0.05% |
6,247 |
26 |
Mozilla/5.0+(compatible;+GrapeshotCrawler / 2.0;++http:/ / www.grapeshot.co.uk/ crawler.php) |
11,004 |
0.04% |
1,163 |
27 |
Mozilla/4.0+(compatible;+MSIE+8.0;+Window s+NT+6.1;+Trident/ 4.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+ 3.5.30729;+.NET+CLR+3.0.30729;+Media+Cent er+PC+6.0;+MDDR;+.NET4.0C;+.NET4.0E;+.NET +CLR+1.1.4322;+Tablet+PC+2.0);+360Spider |
9,881 |
0.04% |
4,896 |
28 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ zh-CN;+rv:1.8.0.11)+Firefox/ 1.5.0.11+360Spider; |
9,816 |
0.04% |
5,880 |
29 |
Mozilla/5.0+(compatible;+008/ 0.85;+http:/ / www.80legs.com/ webcrawler.html)+Gecko/ 2008032620 |
9,787 |
0.04% |
317 |
30 |
Mozilla/5.0+(compatible;+YYSpider;++http: / / www.yunyun.com/ spider.html) |
9,177 |
0.04% |
45 |
31 |
BOT/0.1+(BOT+for+JCE) |
9,038 |
0.04% |
545 |
32 |
Mozilla/5.0+(compatible;+linkdexbot/ 2.0;++http:/ / www.linkdex.com/ about/ bots/ ) |
8,940 |
0.03% |
3,166 |
33 |
Zing-BottaBot/2.0 |
7,227 |
0.03% |
0 |
34 |
Mozilla/5.0+(compatible;+AMZNKAssocBot/ 4.0++http:/ / affiliate-program.amazon.com) |
5,384 |
0.02% |
91 |
35 |
Alexa crawler |
4,816 |
0.02% |
283 |
36 |
Mozilla/5.0+(compatible;+YandexImageResiz er/ 2.0;++http:/ / yandex.com/ bots) |
4,763 |
0.02% |
3,972 |
37 |
Mozilla/5.0+(X11;+Linux+x86_64)+AppleWebK it/ 537.4+(KHTML,+like+Gecko;+Google+Page+Spe ed+Insights)+Chrome/ 22.0.1229+Safari/ 537.4+GoogleBot/ 2.1 |
4,424 |
0.02% |
4 |
38 |
Whitevector+Crawler+(+http:/ / www.whitevector.com/ crawler) |
4,334 |
0.02% |
994 |
39 |
DoCoMo Spider |
3,966 |
0.02% |
167 |
40 |
Google Feedfetcher |
3,526 |
0.01% |
1,806 |
41 |
Mozilla/5.0+(compatible;+ZumBot/ 1.0;+http:/ / help.zum.com/ inquiry) |
3,388 |
0.01% |
13 |
42 |
Mozilla/5.0+(compatible;+SISTRIX+Crawler; +http:/ / crawler.sistrix.net/ ) |
2,679 |
0.01% |
36 |
43 |
piplbot |
2,658 |
0.01% |
126 |
44 |
MSIECrawler |
2,282 |
0.01% |
18 |
45 |
Googlebot |
2,214 |
0.01% |
55 |
46 |
Voila.FR Spider |
1,790 |
0.01% |
1,441 |
47 |
Twitterbot/1.0 |
1,671 |
0.01% |
691 |
48 |
Mozilla/5.0+(compatible;+PaperLiBot/ 2.1;+http:/ / support.paper.li/ entries/ 20023257-what-is-paper-li) |
1,620 |
0.01% |
1,148 |
49 |
SeznamBot/3.0+(+http:/ / fulltext.sblog.cz/ ) |
1,577 |
0.01% |
23 |
50 |
Zing-BottaBot/1.0 |
1,426 |
0.01% |
201 |
51 |
Mozilla/5.0+(compatible;+UnisterBot;+craw ler@unister.de) |
1,332 |
0.01% |
55 |
52 |
PSbot Spider |
1,330 |
0.01% |
569 |
53 |
Mozilla/5.0+(compatible;+CompSpyBot/ 1.0;++http:/ / www.compspy.com/ spider.html) |
1,306 |
0.01% |
13 |
54 |
hivaBot/hivaBot-0.2 |
1,228 |
0.00% |
101 |
55 |
Mozilla/5.0+(compatible;+TweetmemeBot/ 3.0;++http:/ / tweetmeme.com/ ) |
1,170 |
0.00% |
740 |
56 |
Mozilla/5.0+(compatible;+SeznamBot/ 3.1-test1;++http:/ / fulltext.sblog.cz/ ) |
1,130 |
0.00% |
21 |
57 |
RSS-reader of Mac OS X |
1,102 |
0.00% |
1,074 |
58 |
Nutraspace+Spider/Nutch-2.2.1 |
1,092 |
0.00% |
46 |
59 |
Mozilla/5.0+(compatible;+PiplBot;++http:/ / www.pipl.com/ bot/ ) |
951 |
0.00% |
945 |
60 |
Mozilla/5.0+(compatible;+YRSpider;++http: / / www.yunyun.com/ SiteInfo.php? r= about) |
934 |
0.00% |
70 |
61 |
googlebot/2.1+(*http:/ / googlebot.com/ bot.html) |
857 |
0.00% |
5 |
62 |
SAMSUNG-SGH-E250/1.0+Profile/ MIDP-2.0+Configuration/ CLDC-1.1+UP.Browser/ 6.2.3.3.c.1.101+(GUI)+MMP/ 2.0+(compatible;+Googlebot-Mobile/ 2.1;++http:/ / www.google.com/ bot.html) |
845 |
0.00% |
17 |
63 |
Mozilla/5.0+(compatible;+MojeekBot/ 0.5;+http:/ / www.mojeek.com/ bot.html) |
804 |
0.00% |
187 |
64 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ en-US;+rv:1.8.1.6)+Gecko/ 20070725+Firefox/ 2.0.0.6+-+James+BOT+-+WebCrawler+http:/ / cognitiveseo.com/ bot.html |
803 |
0.00% |
241 |
65 |
Mozilla/5.0+(compatible;+TweetedTimes+Bot / 1.0;++http:/ / tweetedtimes.com) |
781 |
0.00% |
633 |
66 |
Yeti/1.0+(NHN+Corp.;+http:/ / help.naver.com/ robots/ ) |
766 |
0.00% |
678 |
67 |
Mozilla/5.0+(compatible;+KaloogaBot;+http :/ / kalooga.com/ crawler) |
753 |
0.00% |
21 |
68 |
mbot+v.1.16 |
718 |
0.00% |
4 |
69 |
IssueCrawler |
651 |
0.00% |
0 |
70 |
SNK+Screenshot+Bot/0.20+(+abuse+at+savenk eep+dot_com+) |
644 |
0.00% |
60 |
71 |
ShowyouBot+(http://showyou.com/ crawler) |
627 |
0.00% |
387 |
72 |
Mozilla/5.0+(iPhone;+CPU+iPhone+OS+6_0_1+ like+Mac+OS+X)+AppleWebKit/ 537.4+(KHTML,+like+Gecko;+Google+Page+Spe ed+Insights)+Version/ 6.0+Mobile/ 10A525+Safari/ 8536.25+GoogleBot/ 2.1 |
598 |
0.00% |
0 |
73 |
Mozilla/5.0+(compatible;+spbot/ 3.1;++http:/ / www.seoprofiler.com/ bot+) |
411 |
0.00% |
192 |
74 |
yacybot+(freeworld/global;+amd64+Windows+ Server+2008+R2+6.1;+java+1.7.0_25;+Europe / de)+http:/ / yacy.net/ bot.html |
408 |
0.00% |
34 |
75 |
Mozilla/5.0+(compatible;+MSIE+9.0;+Window s+NT+5.0;.NET+CLR+1.0.3705;+ContextAd+Bot +1.0) |
400 |
0.00% |
183 |
76 |
Soso spider |
379 |
0.00% |
117 |
77 |
Mozilla/5.0+(compatible;+YIOOP;++http:/ / localhost/ ~lookformedical/ yioop/ bot.php) |
375 |
0.00% |
24 |
78 |
Mozilla/5.0+(compatible;+uMBot-LN/ 1.0;+mailto:+crawling@ubermetrics-technol ogies.com) |
372 |
0.00% |
87 |
79 |
YisouSpider |
363 |
0.00% |
272 |
80 |
TinEye/1.1+(http://tineye.com/ crawler.html) |
321 |
0.00% |
68 |
81 |
MaxPointCrawler/Nutch-1.1+(maxpoint.crawl er+at+maxpointinteractive+dot+com) |
302 |
0.00% |
97 |
82 |
GarlikCrawler/1.2+(http:/ / garlik.com/ ,+crawler@garlik.com) |
292 |
0.00% |
168 |
83 |
Mozilla/5.0+(compatible;+SEO-crawler/ 1.0;) |
288 |
0.00% |
0 |
84 |
GoogleBot/2.1+(+http:/ / www.google.com/ bot.html) |
283 |
0.00% |
280 |
85 |
ContextAd+Bot+1.0 |
282 |
0.00% |
77 |
86 |
adbeat_bot |
279 |
0.00% |
252 |
87 |
Mozilla/5.0+(Windows+NT+6.1;+WOW64;+rv:23 .0)+Gecko/ 20100101+Firefox/ 23.0;+MagicGBot |
254 |
0.00% |
4 |
88 |
AdsBot-Google+(+http:/ / www.google.com/ adsbot.html) |
242 |
0.00% |
93 |
89 |
Mozilla/5.001+(windows;+U;+NT4.0;+en-US;+ rv:1.0)+Gecko/ 25250101+-+crawler@dhoondho.com |
240 |
0.00% |
119 |
90 |
Gigabot Spider |
239 |
0.00% |
42 |
91 |
Mozilla/4.0+(compatible;+MSIE+8.0;+Window s+NT+6.0+;+Claritybot) |
231 |
0.00% |
39 |
92 |
AppEngine-Google;+(+http:/ / code.google.com/ appengine;+appid:+s~stremor-crawler) |
223 |
0.00% |
131 |
93 |
Mozilla/5.0+(compatible;+Netseer+crawler/ 2.0;++http:/ / www.netseer.com/ crawler.html;+crawler@netseer.com) |
223 |
0.00% |
76 |
94 |
QuerySeekerSpider+(+http:/ / queryseeker.com/ bot.html+) |
212 |
0.00% |
64 |
95 |
Mozilla/5.0+(Linux;u;Android+2.3.7;zh-cn; )+AppleWebKit/ 533.1+(KHTML,like+Gecko)+Version/ 4.0+Mobile+Safari/ 533.1+(compatible;++http:/ / www.baidu.com/ search/ spider.html) |
209 |
0.00% |
142 |
96 |
SkimBot/1.0+(www.skimlinks.com+<dev@skiml inks.com>) |
186 |
0.00% |
186 |
97 |
Mozilla/5.0+(compatible;+Dow+Jones+Search bot) |
180 |
0.00% |
1 |
98 |
Mozilla/5.0+(compatible;+YandexDirect/ 3.0;++http:/ / yandex.com/ bots) |
180 |
0.00% |
0 |
99 |
Googlebot-Video/1.0 |
173 |
0.00% |
0 |
100 |
Mozilla/5.0+(compatible;+LauschenBot/ 1.2;+http:/ / www.thoughtboiler.com/ bots) |
156 |
0.00% |
29 |
101 |
Fireball.de Spider |
152 |
0.00% |
3 |
102 |
Alexa Spider |
151 |
0.00% |
136 |
103 |
Screaming+Frog+SEO+Spider/ 2.20 |
151 |
0.00% |
16 |
104 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ en-US;+rv:1.9.1.2)+Gecko/ 20090729+Firefox/ 3.5.2+(.NET+CLR+3.5.30729;+Diffbot/ 0.1;++http:/ / www.diffbot.com) |
144 |
0.00% |
29 |
105 |
Mozilla/5.0+(compatible;+uMBot-FC/ 1.0;+mailto:+crawling@ubermetrics-technol ogies.com) |
140 |
0.00% |
76 |
106 |
spider |
132 |
0.00% |
35 |
107 |
ICC-Crawler/2.0+(Mozilla-compatible;+;+ht tp:/ / kc.nict.go.jp/ project1/ crawl.html) |
129 |
0.00% |
44 |
108 |
Mozilla/4.0+(compatible;+MSIE+6.0;+Window s+NT+5.1;+SV1;++http:/ / www.changedetection.com/ bot.html+) |
124 |
0.00% |
62 |
109 |
Mozilla/5.0+(compatible;+news+bot+/ 2.1) |
121 |
0.00% |
21 |
110 |
Mozilla/4.0+(compatible;+MSIE+6.0;+Window s+NT+5.1;+SV1;+CrystalSemanticsBot+http:/ / www.crystalsemantics.com/ user-agent/ ) |
114 |
0.00% |
111 |
111 |
Mozilla/5.0+(compatible;+Plukkie/ 1.5;+http:/ / www.botje.com/ plukkie.htm) |
104 |
0.00% |
15 |
112 |
Mozilla/5.0+(iPhone;+U;+CPU+iPhone+OS+4_1 +like+Mac+OS+X;+en-us)+AppleWebKit/ 532.9+(KHTML,+like+Gecko)+Version/ 4.0.5+Mobile/ 8B117+Safari/ 6531.22.7+(compatible;+Mediapartners-Goog le/ 2.1;++http:/ / www.google.com/ bot.html) |
103 |
0.00% |
1 |
113 |
Mozilla/xintellibot |
100 |
0.00% |
86 |
114 |
Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10 _8_2)+AppleWebKit/ 537.13+(KHTML,+like+Gecko)+Chrome/ 24.0.1290.2+Safari/ 537.13+Lumibot/ 2.0 |
100 |
0.00% |
0 |
115 |
Willow+Internet+Crawler+by+Twotrees+V2.1 |
92 |
0.00% |
47 |
116 |
Mozilla/5.0+(compatible;+parsijoo-bot;++h ttp:/ / www.parsijoo.ir/ ;+ehsan.mousakazemi@gmail.com) |
91 |
0.00% |
90 |
117 |
Mozilla/5.0+(compatible;+CareerBot/ 1.1;++http:/ / www.career-x.de/ bot.html) |
90 |
0.00% |
36 |
118 |
Phantom.js+bot |
71 |
0.00% |
9 |
119 |
Mozilla/5.0+(compatible;+200PleaseBot/ 1.0;++http:/ / www.200please.com/ bot) |
70 |
0.00% |
35 |
120 |
HubSpot+Links+Crawler+1.0+http:/ / www.hubspot.com/ |
68 |
0.00% |
0 |
121 |
rogerbot/1.0+(http:// moz.com/ help/ pro/ what-is-rogerbot-,+rogerbot-crawler+shiny @moz.com) |
68 |
0.00% |
34 |
122 |
Mozilla/5.0+(compatible;+GeliyooBot/ 1.0beta;++http:/ / www.geliyoo.com/ ) |
62 |
0.00% |
39 |
123 |
ZumBot/1.0+(ZUM+Search;+http:/ / help.zum.com/ inquiry) |
56 |
0.00% |
25 |
124 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ en;+rv:1.9.0.13)+Gecko/ 2009073022+Firefox/ 3.5.2+(.NET+CLR+3.5.30729)+SurveyBot/ 2.3+(DomainTools) |
52 |
0.00% |
18 |
125 |
Mozilla/5.0+(compatible;+008/ 0.83;+http:/ / www.80legs.com/ webcrawler.html;)+Gecko/ 2008032620 |
51 |
0.00% |
42 |
126 |
Mozilla/5.0+(Linux;+U;+Android+4.2.1;+en- us;+CUBOT+GT99+Build/ JOP40D)+AppleWebKit/ 534.30+(KHTML,+like+Gecko)+Version/ 4.0+Mobile+Safari/ 534.30 |
49 |
0.00% |
2 |
127 |
calculon+spider/0.1.0 |
48 |
0.00% |
34 |
128 |
Mozilla/5.0+(Linux;+U;+Android+4.2.2;+en- us;+CUBOT+GT72E+Build/ JDQ39)+AppleWebKit/ 534.30+(KHTML,+like+Gecko)+Version/ 4.0+Mobile+Safari/ 534.30 |
48 |
0.00% |
2 |
129 |
AOL Sureseeker search plugin |
47 |
0.00% |
33 |
130 |
Mozilla/5.0+(Linux;+U;+Android+4.2.1;+cs- cz;+CUBOT+GT99+Build/ JOP40D)+AppleWebKit/ 534.30+(KHTML,+like+Gecko)+Version/ 4.0+Mobile+Safari/ 534.30 |
46 |
0.00% |
1 |
131 |
Mozilla/5.0+(compatible;+heritrix/ 1.14.4++http:/ / netarkivet.dk/ webcrawler/ ) |
45 |
0.00% |
5 |
132 |
Googlebot-Mobile |
43 |
0.00% |
0 |
133 |
Mozilla/5.0+(Windows;+U;+Windows+NT+6.0;+ en-GB;+rv:1.0;+trendictionbot0.5.0;+trend iction+search;+http:/ / www.trendiction.de/ bot;+please+let+us+know+of+any+problems;+ web+at+trendiction.com)+Gecko/ 20071127+Firefox/ 3.0.0.11 |
40 |
0.00% |
20 |
134 |
Wotbox/2.01+(+http:// www.wotbox.com/ bot/ ) |
38 |
0.00% |
13 |
135 |
Mozilla/5.0+(Linux;+Android+4.2.1;+CUBOT+ GT99+Build/ JOP40D)+AppleWebKit/ 535.19+(KHTML,+like+Gecko)+Chrome/ 18.0.1025.166+Mobile+Safari/ 535.19 |
38 |
0.00% |
3 |
136 |
YodaoBot Spider |
38 |
0.00% |
37 |
137 |
BacklinkCrawler+(http:/ / www.backlinktest.com/ crawler.html) |
36 |
0.00% |
1 |
138 |
Mozilla/5.0+(compatible;+Linux;+Socialrad arbot/ 2.0;+en-US;+bot@infegy.com) |
36 |
0.00% |
18 |
139 |
SEOENGWorldBot/1.0+(+http:/ / www.seoengine.com/ seoengbot.htm) |
34 |
0.00% |
14 |
140 |
Mozilla/5.0+(compatible;+Kraken/ 0.1;+http:/ / linkfluence.net/ ;+bot@linkfluence.net) |
32 |
0.00% |
19 |
141 |
Backlink-Check.de+(+http:/ / www.backlink-check.de/ bot.html) |
32 |
0.00% |
8 |
142 |
bitlybot |
30 |
0.00% |
11 |
143 |
PercolateCrawler/3.1.30+(ops@percolate.co m) |
29 |
0.00% |
0 |
144 |
Mozilla/5.0+(compatible;+oBot/ 2.3.1;++http:/ / filterdb.iss.net/ crawler/ ) |
29 |
0.00% |
6 |
145 |
MSRA+Crawler/1.0 |
29 |
0.00% |
21 |
146 |
ExB+Language+Crawler+2.1.5+(+http:/ / www.exb.de/ crawler) |
27 |
0.00% |
27 |
147 |
Aboundex/0.2+(http:// www.aboundex.com/ crawler/ ) |
26 |
0.00% |
2 |
148 |
yacybot+(domengo-global;+amd64+Linux+3.2. 0-4-amd64;+java+1.6.0_27;+Europe/ de)+http:/ / yacy.net/ bot.html |
26 |
0.00% |
19 |
149 |
R6_FeedFetcher_(www.radian6.com/ crawler) |
26 |
0.00% |
8 |
150 |
yacybot+(freeworld/global;+x86+Windows+XP +5.1;+java+1.7.0_25;+America/ es)+http:/ / yacy.net/ bot.html |
26 |
0.00% |
0 |
151 |
Mozilla/5.0+(compatible;+newslebot/ 1.0;++http:/ / newsle.com) |
25 |
0.00% |
19 |
152 |
Mozilla/5.0+(compatible;+meanpathbot/ 1.0;++http:/ / www.meanpath.com/ meanpathbot.html) |
23 |
0.00% |
4 |
153 |
Mozilla/5.0+(compatible;+YandexFavicons/ 1.0;++http:/ / yandex.com/ bots) |
22 |
0.00% |
10 |
154 |
Mozilla/5.0+(compatible;+NetSeer+crawler/ 2.0;++http:/ / www.netseer.com/ crawler.html;+crawler@netseer.com) |
22 |
0.00% |
13 |
155 |
Mozilla/5.0+(compatible;+MSIE+8.0;+Window s+NT+5.1)+KomodiaBot/ 1.0 |
22 |
0.00% |
20 |
156 |
R6_CommentReader(www.radian6.com/ crawler) |
22 |
0.00% |
22 |
157 |
GG+PeekBot+2.0+(+http:/ / gg.pl/ +http:/ / info.gadu-gadu.pl/ praca+) |
21 |
0.00% |
18 |
158 |
DLE_Spider.exe |
21 |
0.00% |
3 |
159 |
Mozilla/5.0+(compatible;+BIXOCRAWLER;++ht tp:/ / wiki.github.com/ bixo/ bixo/ bixocrawler;+bixo-dev@yahoogroups.com) |
20 |
0.00% |
15 |
160 |
Cliqzbot/0.1+(+http:/ / cliqz.com++cliqzbot@cliqz.com) |
19 |
0.00% |
8 |
161 |
Mozilla/5.0+(compatible;+aiHitBot/ 2.8;++http:/ / endb-consolidated.aihit.com/ ) |
18 |
0.00% |
3 |
162 |
My+Nutch+Spider/Nutch-1.7 |
18 |
0.00% |
9 |
163 |
Screaming+Frog+SEO+Spider/ 2.21 |
17 |
0.00% |
5 |
164 |
rim_spider |
16 |
0.00% |
7 |
165 |
User-Agent:+Curious+George+-+www.analytic sseo.com/ crawler |
16 |
0.00% |
11 |
166 |
W3C HTML-Code Validator |
16 |
0.00% |
13 |
167 |
WikiDo/1.1+(http://wikido.com;+crawler@wi kido.com) |
16 |
0.00% |
5 |
168 |
yacybot+(freeworld/global;+amd64+Linux+3. 2.0-4-amd64;+java+1.6.0_27;+Europe/ en)+http:/ / yacy.net/ bot.html |
16 |
0.00% |
14 |
169 |
gsa-crawler+(Enterprise;+GID-01422;+jplas tiras@google.com) |
15 |
0.00% |
2 |
170 |
Mozilla/5.0+(compatible;+pub-crawler;++ht tp:/ / wiki.github.com/ bixo/ bixo/ bixocrawler;+bixo-dev@yahoogroups.com) |
15 |
0.00% |
6 |
171 |
Mozilla/5.0+(compatible;+oBot/ 2.3.1;+http:/ / filterdb.iss.net/ crawler/ ) |
15 |
0.00% |
5 |
172 |
Y!J-BRJ/YATS+crawler+(http:/ / help.yahoo.co.jp/ help/ jp/ search/ indexing/ indexing-15.html) |
14 |
0.00% |
7 |
173 |
Mozilla/5.0+(compatible;+Nigma.ru/ 3.0;+crawler@nigma.ru) |
14 |
0.00% |
7 |
174 |
LinkedInBot/1.0+(compatible;+Mozilla/ 5.0;+Jakarta+Commons-HttpClient/ 3.1++http:/ / www.linkedin.com) |
14 |
0.00% |
7 |
175 |
DuckDuckBot/1.1;+(+http:/ / duckduckgo.com/ duckduckbot.html) |
14 |
0.00% |
12 |
176 |
SupremeSearch.net+Spider-Bot/ 5.5+(http:/ / www.supremesearch.net) |
14 |
0.00% |
7 |
177 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;f r;rv:1.8.1)+VoilaBotCollector+BETA+0.1++( http:/ / www.voila.com/ ) |
14 |
0.00% |
14 |
178 |
Oxyme.Search+-+Web+crawler |
14 |
0.00% |
10 |
179 |
Mozilla/5.0+(compatible;+pmoz.info+ODP+li nk+checker;++http:/ / pmoz.info/ doc/ botinfo.htm) |
13 |
0.00% |
8 |
180 |
Mozilla/5.0+(compatible;+archive.org_bot; +Wayback+Machine+Live+Record;++http:/ / archive.org/ details/ archive.org_bot) |
13 |
0.00% |
13 |
181 |
Mozilla/5.0+(compatible;+LuminateBot/ 1.0;++http:/ / www.luminate.com/ bot/ ) |
13 |
0.00% |
6 |
182 |
yacybot+(freeworld/global;+amd64+Linux+3. 2.0-4-amd64;+java+1.7.0_25;+America/ en)+http:/ / yacy.net/ bot.html |
13 |
0.00% |
11 |
183 |
Mozilla/5.0+(compatible;+archive.org_bot+ +http:/ / archive.org/ details/ archive.org_bot) |
12 |
0.00% |
5 |
184 |
Mozilla/4.0+(compatible;++Vagabondo/ 4.0;+webcrawler+at+wise-guys+dot+nl;+http :/ / webagent.wise-guys.nl/ ;+http:/ / www.wise-guys.nl/ ) |
12 |
0.00% |
2 |
185 |
crawler4j+(http://code.google.com/ p/ crawler4j/ ) |
12 |
0.00% |
11 |
186 |
Cliqzbot/0.1+(+http:/ / cliqz.com/ company/ cliqzbot) |
12 |
0.00% |
9 |
187 |
yacybot+(freeworld/global;+x86+Windows+7+ 6.1;+java+1.7.0_17;+Asia/ zh)+http:/ / yacy.net/ bot.html |
11 |
0.00% |
1 |
188 |
SppcBot/1 |
11 |
0.00% |
7 |
189 |
Mozilla/4.0+(compatible;+Crawler;+MSIE+7. 0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.5072 7) |
10 |
0.00% |
10 |
190 |
ShopWiki/1.0+(++http:/ / www.shopwiki.com/ wiki/ Help:Bot) |
10 |
0.00% |
5 |
191 |
Tweetbot/28500+CFNetwork/ 672.0.2+Darwin/ 14.0.0 |
10 |
0.00% |
10 |
192 |
AdsBot-Google-Mobile+(+http:/ / www.google.com/ mobile/ adsbot.html)+Mozilla+(iPhone;+U;+CPU+iPho ne+OS+3+0+like+Mac+OS+X)+AppleWebKit+(KHT ML,+like+Gecko)+Mobile+Safari |
10 |
0.00% |
4 |
193 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ en-US;+rv:1.8.1.6)+Gecko/ 20070725+Firefox/ 2.0.0.6+Ara.com.tr+AraBot+1.0 |
10 |
0.00% |
1 |
194 |
FyberSpider/1.3+(http:/ / www.fybersearch.com/ fyberspider.php) |
10 |
0.00% |
5 |
195 |
Mozilla/5.0+(compatible;+metager2-verific ation-bot;++http:/ / metager2.de/ technology.php) |
10 |
0.00% |
7 |
196 |
Mozilla/5.0+(compatible;+archive.org_bot; +Archive-It;++http:/ / archive-it.org/ files/ site-owners.html) |
10 |
0.00% |
6 |
197 |
Mozilla/5.0+(compatible;+sharegloo_bot/ 1.0;++http:/ / www.sharegloo.com) |
9 |
0.00% |
8 |
198 |
Semantifire1/0.20+(+http:/ / www.setooz.com/ oozbot.html+;+agentname+at+setooz+dot_com +) |
9 |
0.00% |
1 |
199 |
datagnionbot+(+http:/ / www.datagnion.com/ bot.html) |
9 |
0.00% |
7 |
200 |
ArtavisBot+1.0 |
9 |
0.00% |
6 |
201 |
Mozilla/5.0+(compatible;+Falconsbot;++htt p:/ / ws.nju.edu.cn/ falcons/ ) |
8 |
0.00% |
3 |
202 |
dotbot |
8 |
0.00% |
4 |
203 |
OdklBot/1.0+(klass@odnoklassniki.ru) |
8 |
0.00% |
6 |
204 |
Optimus5+Spider |
8 |
0.00% |
6 |
205 |
Elisabot+1.0+unforgettable |
8 |
0.00% |
2 |
206 |
bitlybot/2.0 |
7 |
0.00% |
5 |
207 |
Speedy Spider |
7 |
0.00% |
2 |
208 |
yacybot+(freeworld/global;+amd64+Linux+3. 2.0-53-generic;+java+1.7.0_40;+Europe/ en)+http:/ / yacy.net/ bot.html |
7 |
0.00% |
3 |
209 |
Mozilla/4.0+(compatible;+BOTW+Spider;++ht tp:/ / botw.org) |
7 |
0.00% |
4 |
210 |
Mozilla/4.0+(CMS+Crawler:+http:/ / www.cmscrawler.com) |
7 |
0.00% |
3 |
211 |
Y!J-BSC/1.0+crawler+(http:/ / help.yahoo.co.jp/ help/ jp/ blog-search/ ) |
7 |
0.00% |
7 |
212 |
ssearch_bot+(sSearch+Crawler;+http:/ / www.semantissimo.de) |
7 |
0.00% |
4 |
213 |
rogerbot/1.0+(http:// www.moz.com/ dp/ rogerbot,+rogerbot-crawler@moz.com) |
7 |
0.00% |
0 |
214 |
Feedjit+Favicon+Crawler+1.0 |
7 |
0.00% |
7 |
215 |
Y!J-BRO/YFSJ+crawler+(compatible;+Mozilla +4.0;+MSIE+5.5;+http:/ / help.yahoo.co.jp/ help/ jp/ search/ indexing/ indexing-15.html;+YahooFeedSeekerJp/ 2.0) |
6 |
0.00% |
4 |
216 |
fastbot.de+crawler+2.0+beta+(http:/ / www.fastbot.de) |
6 |
0.00% |
6 |
217 |
Yeti/1.1+(NHN+Corp.;+http:/ / help.naver.com/ robots/ ) |
6 |
0.00% |
3 |
218 |
Curious+George+-+www.analyticsseo.com/ crawler |
6 |
0.00% |
4 |
219 |
Mozilla/5.0+(compatible;+adbeat-publisher -description-fetcher;++crawler@scaleunlim ited.com;+crawler@scaleunlimited.com) |
6 |
0.00% |
3 |
220 |
Mozilla/5.0+(Windows;+U;+Windows+NT+6.0;+ en-US;+rv:1.9.0.4)+Gecko/ 2008102920+http:/ / ow.ly+web+crawler+(.NET+CLR+3.5.30729) |
6 |
0.00% |
3 |
221 |
Adknowledge+Engage+Crawler |
6 |
0.00% |
6 |
222 |
Screaming+Frog+SEO+Spider/ 2.11 |
6 |
0.00% |
1 |
223 |
TinEye-bot/0.02+(see+http:/ / www.tineye.com/ crawler.html) |
5 |
0.00% |
4 |
224 |
Search4free Spider |
5 |
0.00% |
2 |
225 |
Xaldon+WebSpider |
5 |
0.00% |
1 |
226 |
Open+Web+Analytics+Bot+1.3.1 |
5 |
0.00% |
5 |
227 |
BuddhaBot |
5 |
0.00% |
2 |
228 |
Mozilla/5.0+(compatible;+dlcbot/ 0.1;++http:/ / www.drlinkcheck.com/ ) |
5 |
0.00% |
5 |
229 |
TasBot+(http://www.httpool.com/ ) |
5 |
0.00% |
5 |
230 |
Who.is+Bot |
4 |
0.00% |
2 |
231 |
yacybot+(freeworld/global;+amd64+Windows+ Server+2008+R2+6.1;+java+1.7.0_04;+Europe / en)+http:/ / yacy.net/ bot.html |
4 |
0.00% |
2 |
232 |
Crawler |
4 |
0.00% |
3 |
233 |
Mozilla/5.0+(compatible;+Infohelfer/ 1.3.3;++http:/ / www.infohelfer.de/ crawler.php) |
4 |
0.00% |
2 |
234 |
AstuteSocialCrawler/Nutch-1.4 |
4 |
0.00% |
2 |
235 |
yacybot+(freeworld/global;+amd64+Linux+3. 4.47-2.38-desktop;+java+1.7.0_40;+Europe/ de)+http:/ / yacy.net/ bot.html |
4 |
0.00% |
4 |
236 |
RavenCrawler |
4 |
0.00% |
3 |
237 |
Mozilla/5.0+(compatible;+NLIL_IAHarvester ++http:/ / www.archive.org/ details/ archive.org_bot) |
4 |
0.00% |
2 |
238 |
SBL-BOT+(http://sbl.net) |
4 |
0.00% |
1 |
239 |
UCMore+Crawler+App |
4 |
0.00% |
0 |
240 |
blogtop.us+crawler+-+http:/ / blogtop.us/ |
4 |
0.00% |
3 |
241 |
Mozilla/5.0+(compatible;+YandexAntivirus/ 2.0;++http:/ / yandex.com/ bots) |
4 |
0.00% |
3 |
242 |
InsieveBot |
4 |
0.00% |
4 |
243 |
Y!J-BRW/1.0+crawler+(http:/ / help.yahoo.co.jp/ help/ jp/ search/ indexing/ indexing-15.html) |
4 |
0.00% |
2 |
244 |
yacybot+(freeworld/global;+amd64+Windows+ 8+6.2;+java+1.7.0_25;+Europe/ de)+http:/ / yacy.net/ bot.html |
3 |
0.00% |
3 |
245 |
JOC+Web+Spider |
3 |
0.00% |
3 |
246 |
Mozilla/5.0+(compatible;+WASALive-Bot+;+h ttp:/ / blog.wasalive.com/ wasalive-bots/ ) |
3 |
0.00% |
3 |
247 |
Mozilla/5.0+(Windows+NT+6.1;+WOW64)+Apple WebKit/ 537.4+(KHTML,+like+Gecko)+Chrome/ 22.0.1229.94+Safari/ 537.4+Innity+Bot |
3 |
0.00% |
1 |
248 |
GermCrawler |
3 |
0.00% |
1 |
249 |
yacybot+(freeworld/global;+amd64+Windows+ 7+6.1;+java+1.7.0_04;+Europe/ de)+http:/ / yacy.net/ bot.html |
3 |
0.00% |
1 |
250 |
Mozilla/5.0+(compatible;+Qualidator.com+B ot+1.0;) |
3 |
0.00% |
1 |
251 |
yacybot+(freeworld/global;+amd64+Linux+3. 2.0-4-amd64;+java+1.7.0_25;+Europe/ en)+http:/ / yacy.net/ bot.html |
3 |
0.00% |
2 |
252 |
AcademicBot+(info@AcademicConcepts.net) |
3 |
0.00% |
1 |
253 |
Toplistbot |
3 |
0.00% |
1 |
254 |
yacybot+(freeworld/global;+amd64+Linux+3. 11.3-gentoo;+java+1.7.0_40;+Europe/ en)+http:/ / yacy.net/ bot.html |
3 |
0.00% |
2 |
255 |
Mozilla/5.0+(compatible;+YioopBot;++http: / / 173.13.143.74/ bot.php) |
3 |
0.00% |
3 |
256 |
Twitterbot/0.1;c2VvLWNyYXdsZXI= |
2 |
0.00% |
2 |
257 |
123peoplebot/1.0 |
2 |
0.00% |
2 |
258 |
Mozilla/5.0+(Linux;+U;+Android+4.2.1;+pt- pt;+CUBOT+ONE+Build/ JOP40D)+AppleWebKit/ 534.30+(KHTML,+like+Gecko)+Version/ 4.0+Mobile+Safari/ 534.30 |
2 |
0.00% |
1 |
259 |
Mozilla+4.0(compatible;+BotSeer/ 1.0;++http:/ / botseer.ist.psu.edu) |
2 |
0.00% |
2 |
260 |
yacybot+(freeworld/global;+x86+Windows+8+ 6.2;+java+1.7.0_40;+Europe/ en)+http:/ / yacy.net/ bot.html |
2 |
0.00% |
1 |
261 |
TweetbotPad/28500+CFNetwork/ 672.0.2+Darwin/ 14.0.0 |
2 |
0.00% |
2 |
262 |
My+Spider+3/Nutch-1.6 |
2 |
0.00% |
1 |
263 |
LinksCrawler+0.1beta |
2 |
0.00% |
1 |
264 |
EMPAS.ROBOT+(compatible;+MSIE+5.01;+Windo ws+NT+5.0);c2VvLWNyYXdsZXI= |
2 |
0.00% |
2 |
265 |
Xerka+WebBot+v1.0.0+[Intercontrol] |
2 |
0.00% |
0 |
266 |
Mozilla/5.0+(compatible;+MarketwireBot/ 1.0;++http:/ / www.marketwire.com) |
2 |
0.00% |
1 |
267 |
Mozilla/5.0+(compatible;+spbot/ 4.0b;++http:/ / www.seoprofiler.com/ bot+) |
2 |
0.00% |
1 |
268 |
Mozilla/5.0+(compatible;+IstellaBot/ 1.10.2++http:/ / www.tiscali.it/ ) |
2 |
0.00% |
1 |
269 |
Mozilla/5.0+(compatible;+SolomonoBot/ 1.05;++http:/ / www.solomono.ru) |
2 |
0.00% |
1 |
270 |
Mioolabot;+(+http://www.mioola.com/ ) |
2 |
0.00% |
1 |
271 |
yacybot+(freeworld/global;+amd64+Linux+3. 5.0-42-generic;+java+1.6.0_27;+US/ en)+http:/ / yacy.net/ bot.html |
2 |
0.00% |
1 |
272 |
Mozilla/5.0+(compatible;+EtaoSpider/ 1.0;+http:/ / open.etao.com/ dev/ EtaoSpider) |
2 |
0.00% |
2 |
273 |
Mozilla/5.0+(compatible;+redditbot/ 1.0;++http:/ / www.reddit.com/ feedback) |
2 |
0.00% |
1 |
274 |
CorenSearchBot/1.7+en+libwww-perl/ 6.04 |
2 |
0.00% |
1 |
275 |
WCH+Web+Spider |
2 |
0.00% |
2 |
276 |
Open+Web+Analytics+Bot+1.5.4 |
2 |
0.00% |
1 |
277 |
CodeGator+Crawler+v1.0 |
2 |
0.00% |
2 |
278 |
yacybot+(webportal-global;+amd64+Linux+3. 5.0-36-generic;+java+1.7.0_40;+Europe/ en)+http:/ / yacy.net/ bot.html |
2 |
0.00% |
2 |
279 |
WordChampBot |
2 |
0.00% |
0 |
280 |
MetaCartaWebCrawler/4.9.0+Beta+(May+24+20 13+23:34:51;+pslater@geoint.com.au) |
2 |
0.00% |
1 |
281 |
ZEEFbot+0.2+(+https:/ / zeef.com/ bot) |
2 |
0.00% |
1 |
282 |
testCrawler/1.0 |
2 |
0.00% |
0 |
283 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.4.3;+http:/ / www.majestic12.co.uk/ bot.php? +) |
2 |
0.00% |
2 |
284 |
QrobotW |
2 |
0.00% |
1 |
285 |
A+.NET+Web+Crawler |
2 |
0.00% |
1 |
286 |
Mozilla/5.0+(compatible;+Linux+x86_64;+Ma il.RU_Bot/ Img/ 2.0;++http:/ / go.mail.ru/ help/ robots) |
2 |
0.00% |
2 |
287 |
NutchCrawler/Nutch-1.6 |
2 |
0.00% |
1 |
288 |
Test+Webbot |
2 |
0.00% |
1 |
289 |
yacybot+(freeworld/global;+amd64+Windows+ 7+6.1;+java+1.7.0_09;+Europe/ de)+http:/ / yacy.net/ bot.html |
2 |
0.00% |
1 |
290 |
Motzilla/4.0(iPad;I;Minix2.4.3;MIPS;rv1.8 )Gangsta/ 20070518Hypnospider/ 0.7 |
1 |
0.00% |
0 |
291 |
AdnormCrawler+www.adnorm.com/ crawler |
1 |
0.00% |
1 |
292 |
Mozilla/5.0+(compatible;+spbot/ 2.1;++http:/ / www.seoprofiler.com/ bot);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
293 |
NutchCVS/0.7.1+(Nutch;+http:/ / lucene.apache.org/ nutch/ bot.html;+nutch-agent@lucene.apache.org) |
1 |
0.00% |
1 |
294 |
Mozilla/5.0+(compatible;+LikaholixCrawler / 1.0;++http:/ / mylikes.com/ about/ crawler) |
1 |
0.00% |
1 |
295 |
yacybot+(freeworld/global;+x86+Windows+7+ 6.1;+java+1.7.0_40;+Europe/ fr)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
296 |
Mozilla/5.0+(compatible;+web-ceo-online-b ot/ 1.0;++http:/ / online.webceo.com) |
1 |
0.00% |
1 |
297 |
Peeplo+Screenshot+Bot/ 0.20+(+abuse+at+peeplo+dot_com+) |
1 |
0.00% |
1 |
298 |
Dalvik/0.8.15(Windoof;I;TuringOS2.4.8;SPA RC64;rv1.2.6)Gangsta/ 20050915Spacebot/ 11.0b5 |
1 |
0.00% |
0 |
299 |
naverbookmarkcrawler |
1 |
0.00% |
1 |
300 |
Mozilla/5.0+(compatible;+spbot/ 2.0.4;++http:/ / www.seoprofiler.com/ bot);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
301 |
Mozilla/5.0(X11;I;Android2.2.12;i586;rv1. 3.10)Salamander/ 20030809Lightningspider/ 0.7 |
1 |
0.00% |
0 |
302 |
ConveraCrawler/0.9d(+http:/ / www.authoritativeweb.com/ crawl) |
1 |
0.00% |
1 |
303 |
Motzilla/4.0(iPad;N;HP-SUX2.4.25;8088;rv1 .5.7)Trident/ 20050915Waterbot/ 3.0.7 |
1 |
0.00% |
0 |
304 |
CowCrawler/CowCrawler-dev+(+http:/ / beta.cow.com);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
305 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.2.4;+http:/ / www.majestic12.co.uk/ bot.php? +);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
306 |
Mozilla/5.0+(compatible;++6889;+jFirewall Bot/ 4.0.61;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
307 |
yacybot(ppcMacOSX10.5.2;java1.5.0_13;Euro pe/ de)http:/ / yacy.net/ bot.html |
1 |
0.00% |
0 |
308 |
yacybot+(freeworld/global;+amd64+Linux+3. 2.0-50-generic;+java+1.6.0_27;+America/ en)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
309 |
MLBot+(www.metadatalabs.com/ mlbot);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
310 |
Godzilla/9.0(Windoof;I;IRIX2.0.16;z196;rv 1.8.9)Alligator/ 20040411Windbot/ 1.5.0.2 |
1 |
0.00% |
0 |
311 |
Tweetbot/28500+CFNetwork/ 609.1.4+Darwin/ 13.0.0 |
1 |
0.00% |
1 |
312 |
Mozilla/4.0+(compatible;+MSIE+6.0;+Window s+NT+5.1;+Googlebot) |
1 |
0.00% |
0 |
313 |
ESISmartSpider;c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
314 |
GentleCrawler |
1 |
0.00% |
0 |
315 |
disco/Nutch-0.9(experimentalcrawler;nedro cks@gmail.com) |
1 |
0.00% |
1 |
316 |
Mozilla/5.0(Windows;U;WindowsNT5.1;en-US; rv:1.9.0.8)Gecko/ 2009032609Googlebot2.1 |
1 |
0.00% |
0 |
317 |
DuckDuckBot/1.1;+(+http:/ / duckduckgo.com/ duckduckbot.html);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
318 |
Mozilla/5.0(Windows;U;WindowsNT6.0;en-US; rv:1.9.2.4)Gecko/ 20100611Googlebot2.1(.NETCLR3.5.30729) |
1 |
0.00% |
1 |
319 |
Godzilla/9.0(Windows;I;AmigaDOS2.2.38;P50 0;rv1.7.8)Trident/ 20080524Firebot/ 10.0a1 |
1 |
0.00% |
0 |
320 |
Mozilla/5.0+(compatible;+geourl/ 3.1+-+http:/ / geourl.org/ bot) |
1 |
0.00% |
1 |
321 |
Mozilla/4.0+(compatible;+BlitzBot);c2VvLW NyYXdsZXI= |
1 |
0.00% |
1 |
322 |
My+Nutch+Spider/Nutch-2.2.1 |
1 |
0.00% |
1 |
323 |
googlebot/1.0 |
1 |
0.00% |
0 |
324 |
Googlebot-2.1 |
1 |
0.00% |
0 |
325 |
dtSearchSpider |
1 |
0.00% |
0 |
326 |
TinEye/1.1+(http://tineye.com/ crawler.html);c2VvLWNyYXdsZXI= |
1 |
0.00% |
0 |
327 |
Godzilla/9.0(OS/2;U;FreeDOS98;ia32;rv1.2. 10)WebKit/ 20050915Sunspider/ 10.0b2 |
1 |
0.00% |
0 |
328 |
Mozilla/5.0+(compatible;++2822;+jFirewall Bot/ 4.0.14;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
329 |
yacybot+(freeworld/global;+x86_64+Mac+OS+ X+10.8.5;+java+1.6.0_51;+Europe/ de)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
330 |
AbachoBOT |
1 |
0.00% |
0 |
331 |
Mozilla/5.0+(compatible;++6133;+jFirewall Bot/ 4.0.45;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
332 |
Mozilla/4.0+(compatible;+MSIE+7.0;+Window s+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0 .50727)+yolinkBot |
1 |
0.00% |
1 |
333 |
OmniExplorer_Bot/5.28+(+http:/ / www.omni-explorer.com)+WorldIndexer;c2VvL WNyYXdsZXI= |
1 |
0.00% |
1 |
334 |
AdsBot-Google-Mobile(Android;+http:/ / www.google.com/ adsbot.html) |
1 |
0.00% |
0 |
335 |
Dalvik/0.8.15(OpenVMS;U;WindoofCE2.4.10;I BM1130;rv1.3)Gangsta/ 20030809Hypnospider/ 3.0.7 |
1 |
0.00% |
0 |
336 |
Godzilla/9.0(VGA;U;SunOS2.2.23;P500;rv9.0 )Alligator/ 20050915Turbospider/ 3.6.23 |
1 |
0.00% |
1 |
337 |
yacybot+(x86+Windows+XP+5.1;+java+1.6.0_1 7;+Europe/ de)+http:/ / yacy.net/ bot.html;c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
338 |
Mozilla/5.0(Windows;U;WindowsNT5.1;en-US; rv:1.9.0.1)Gecko/ 2008070208Googlebot2.1 |
1 |
0.00% |
0 |
339 |
ScoutJet web crawler |
1 |
0.00% |
1 |
340 |
Mozilla/4.0+(compatible;+Struct.it+crawle r;++http:/ / struct.it/ contacts/ ) |
1 |
0.00% |
1 |
341 |
yacybot+(freeworld/global;+i386+Linux+2.6 .32-5-686;+java+1.6.0_27;+Europe/ en)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
342 |
FlightDeckReportsBot/ 2.0+(http:/ / www.flightdeckreports.com/ pages/ bot) |
1 |
0.00% |
1 |
343 |
NEC Spider |
1 |
0.00% |
1 |
344 |
Mozilla/5.0+(compatible;++7442;+jFirewall Bot/ 4.0.75;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
345 |
Mozilla/5.0+(compatible;++9944;+jFirewall Bot/ 4.0.14;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
0 |
346 |
Mozilla/5.0+(compatible;++8779;+jFirewall Bot/ 4.0.99;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
347 |
Mozilla/5.0(Windows;U;WindowsNT5.1;en-GB; rv:1.9.0.8)Gecko/ 2009032609Googlebot2.1 |
1 |
0.00% |
0 |
348 |
Mozilla/5.0+(compatible;++3964;+jFirewall Bot/ 4.0.71;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
349 |
Dalvik/0.8.15(Prime;U;webOS2.0.30;DSP5680 0;rv1.7)Gecko/ 20120101Seabot/ 2.0.0.8 |
1 |
0.00% |
0 |
350 |
grokkit-crawler+(pdsupport@purediscovery. com) |
1 |
0.00% |
1 |
351 |
Dalvik/0.8.15(Motif;U;FreeDOS2.0.14;CDC86 00;rv1.5.4)Gecko/ 20120101Seabot/ 2.0.0.2 |
1 |
0.00% |
0 |
352 |
DomainCrawler/1.0(info@domaincrawler.com; http:/ / www.domaincrawler.com/ domains/ view/ appointmentnet.com) |
1 |
0.00% |
0 |
353 |
Mozilla/5.0(Windows;U;WindowsNT5.1;en-US; rv:1.9.0.4)Gecko/ 2008102920googlebot |
1 |
0.00% |
0 |
354 |
Mozilla/5.0+(compatible;++4330;+jFirewall Bot/ 4.0.65;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
0 |
355 |
yacybot+(freeworld/global;+amd64+Windows+ 7+6.1;+java+1.7.0_40;+America/ en)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
356 |
Goglebot/2.1(+http:// www.goglebot.com/ bot.html) |
1 |
0.00% |
0 |
357 |
Mozilla/5.0+(compatible;+AbiLogicBot/ 1.0;++http:/ / www.abilogic.com);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
358 |
Mozilla/5.0+(compatible;+Dow+Jones+Search bot);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
359 |
Mozilla/5.0+(compatible;++8259;+jFirewall Bot/ 4.0.74;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
0 |
360 |
googlebot/2.X(+http:/ / www.google.com/ bot.html) |
1 |
0.00% |
0 |
361 |
Mozilla/5.0+(Linux;+U;+Android+4.2.1;+hr- hr;+CUBOT+ONE+Build/ JOP40D)+AppleWebKit/ 534.30+(KHTML,+like+Gecko)+Version/ 4.0+Mobile+Safari/ 534.30 |
1 |
0.00% |
1 |
362 |
Mozilla/5.0+(compatible;++1027;+jFirewall Bot/ 4.0.95;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
363 |
Mozilla/5.0+(compatible;++5917;+jFirewall Bot/ 4.0.69;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
0 |
364 |
Mozilla/5.0+(X11;+Linux+x86_64)+AppleWebK it/ 534.34+(KHTML,+like+Gecko)+TarantulaBot+( Linux+x86_64)+Webkit/ Apple+Safari/ 1.5+Safari/ 534.34 |
1 |
0.00% |
1 |
365 |
Mozilla/5.0+(compatible;++2247;+jFirewall Bot/ 4.0.57;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
366 |
everyfeed-spider/2.0(http:/ / www.everyfeed.com) |
1 |
0.00% |
0 |
367 |
yacybot+(freeworld/global;+amd64+Linux+2. 6.32-5-amd64;+java+1.6.0_27;+Europe/ en)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
368 |
yacybot(freeworld/global;amd64Windows76.1 ;java1.6.0_26;Europe/ fr)http:/ / yacy.net/ bot.html |
1 |
0.00% |
0 |
369 |
Hello+from+Axtaris-Crawler!+(http:/ / axtaris.org) |
1 |
0.00% |
1 |
370 |
Rankerbot/0.1 |
1 |
0.00% |
1 |
371 |
Dalvik/0.8.15(iPad;I;webOS2.2.23;MIPS;rv1 .7.5)Penguin/ 20110101Seabot/ 3.5.15 |
1 |
0.00% |
0 |
372 |
archive.org_bot |
1 |
0.00% |
0 |
373 |
X3Watch+Crawler+(http:/ / x3watch.com) |
1 |
0.00% |
1 |
374 |
Godzilla/9.0(Linux;N;WindoofNT2.0.35;AVR3 2;rv1.6.2)Crocodile/ 20050915Firespider/ 8.0.1 |
1 |
0.00% |
0 |
375 |
DomainCrawler/1.0(info@domaincrawler.com; http:/ / www.domaincrawler.com/ domains/ view/ localculinaryschools.com) |
1 |
0.00% |
0 |
376 |
Mozilla/5.0+(compatible;+Linkody+bot+0.2) |
1 |
0.00% |
1 |
377 |
Mozilla/5.0+(compatible;++1432;+jFirewall Bot/ 4.0.46;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
0 |
378 |
Godzilla/9.0(Atari;N;EncomOS2.6.25;PIC18; rv1.5.7)WebKit/ 20090530Turbospider/ 0.5 |
1 |
0.00% |
1 |
379 |
Crawler/1.0+http://elibron.com |
1 |
0.00% |
0 |
380 |
GarlikCrawler/1.1(http:/ / garlik.com/ ,crawler@garlik.com) |
1 |
0.00% |
0 |
381 |
Dalvik/0.8.15(Winblows;N;WindowsNT2.0.4;E N12-82;rv1.7.11)WebKit/ 20040411Windspider/ 3.0.6 |
1 |
0.00% |
1 |
382 |
icerocket web crawler |
1 |
0.00% |
0 |
383 |
Mozilla/5.0+(compatible;++8795;+jFirewall Bot/ 4.0.3;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
384 |
Godzilla/9.0(X11;I;AIX2.4.22;OISC/ E25;rv1.5.1)Crocodile/ 20050915Turbospider/ 3.5.14 |
1 |
0.00% |
0 |
385 |
SolomonoBot/1.01+(http:/ / www.solomono.ru);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
386 |
IP2MapBot/1.3<ahref= http:/ / www.ip2map.com& gt;http:/ / www.ip2map.com& lt;/ a& gt; |
1 |
0.00% |
0 |
387 |
DataSpearSpiderBot/0.2(DataSpearSpiderBot ;http:/ / dssb.dataspear.com/ bot.html;dssb@dataspear.com) |
1 |
0.00% |
0 |
388 |
yacybot+(freeworld/global;+amd64+Windows+ 7+6.1;+java+1.6.0_31;+Europe/ nl)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
389 |
Mozilla/5.0+(compatible;+YesupBot/ 1.0;++http:/ / www.yesup.net/ bot.html) |
1 |
0.00% |
1 |
390 |
dpdev/Nutch-1.0-dev(datapatrolfromgarlik. com;http:/ / www.garlik.com/ crawler;crawleratgarlikdotcom) |
1 |
0.00% |
0 |
391 |
Dalvik/0.8.15(OpenVMS;I;SunOS2.6.27;C90;r v1.4.10)Gecko/ 20090530Webspider/ 2.0.0.16 |
1 |
0.00% |
0 |
392 |
DiBot+Java/1.5.0_04 |
1 |
0.00% |
1 |
393 |
Motzilla/5.0(iPad;U;GNU/ Hurd2.6.16;x86;rv1.6.3)KHTML/ 20030809Webbot/ 11.0a1 |
1 |
0.00% |
0 |
394 |
Mozilla/4.0+(compatible;+MSIE+7.0;+Window s+NT+5.1;+Crayon+Crawler;+.NET+CLR+1.1.43 22);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
395 |
Mozilla/5.0(Windows;U;WindowsNT6.0;en-US; rv:1.9.0.2)Gecko/ 2008091620Googlebot2.1 |
1 |
0.00% |
0 |
396 |
Dalvik/0.8.15(iPod;N;MS-DOS2.6.34;i386;rv 1.7.7)WebKit/ 20070518Seabot/ 2.0.0.3 |
1 |
0.00% |
0 |
397 |
yacybot+(freeworld/global;+amd64+Linux+3. 10.7-gentoo-r1;+java+1.7.0_40;+Europe/ de)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
398 |
DomainCrawler/2.0(info@domaincrawler.com; http:/ / www.domaincrawler.com/ localtruckingschools.com |
1 |
0.00% |
0 |
399 |
Mozilla/5.0+(compatible;+aiHitBot/ 1.0;++http:/ / www.aihit.com/ );c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
400 |
Motzilla/5.0(iPod;U;Android2.4.2;MIPS64;r v1.6.11)KHTML/ 20061222Icebot/ 1.0.6 |
1 |
0.00% |
0 |
401 |
ReutersCrawler/1.0+(http:/ / reuters.com/ crawler.html);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
402 |
Wisenut Spider |
1 |
0.00% |
1 |
403 |
Mozilla/5.0+(compatible;+008/ 0.83;+http:/ / www.80legs.com/ webcrawler.html)+Gecko/ 2008032620;c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
404 |
Mozilla/5.0+(compatible;+MJ12bot/ v1.2.3;+http:/ / www.majestic12.co.uk/ bot.php? +);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
405 |
Googoebot/2.1 |
1 |
0.00% |
0 |
406 |
aardvark-crawler |
1 |
0.00% |
1 |
407 |
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+ en-US;+rv:1.9.1.2)+Gecko/ 20090729+Firefox/ 3.5.2+(.NET+CLR+3.5.30729;+Diffbot/ 0.1;++http:/ / www.diffbot.com);c2VvLWNyYXdsZXI= |
1 |
0.00% |
1 |
408 |
Dalvik/0.8.15(iPod;N;Linux2.2.27;Alpha;rv 1.3)Salamander/ 20110101Windspider/ 3.0.19 |
1 |
0.00% |
1 |
409 |
sproose/0.1+(sproose+bot;+http:/ / www.sproose.com/ bot.html;+crawler@sproose.com) |
1 |
0.00% |
1 |
410 |
Jabse.com+Crawler+v.1.0+www.jabse.com/ crawler.php/ / imagecrawler |
1 |
0.00% |
1 |
411 |
Mozilla/5.0+(compatible;+PerfectMarketKWT bot/ 1.0++http:/ / www.perfectmarket.com) |
1 |
0.00% |
1 |
412 |
ExcaliburInternetSpiderV6.5.4 |
1 |
0.00% |
1 |
413 |
googlebot/2.1(+http:/ / www.googlebot.com/ bot.html |
1 |
0.00% |
0 |
414 |
NutchCVS/0.7.1(NutchrunningatUW;http:/ / www.nutch.org/ docs/ en/ bot.html;sycrawl@cs.washington.edu) |
1 |
0.00% |
1 |
415 |
Mozilla/5.0+(Linux;+U;+Android+3.0.1;+en- us;+Bottle+of+Smoke+Build/ HRI66)+AppleWebKit/ 534.13+(KHTML,+like+Gecko)+Version/ 4.0+Safari/ 534.13 |
1 |
0.00% |
1 |
416 |
mediawords+bot+(http:/ / cyber.law.harvard.edu) |
1 |
0.00% |
1 |
417 |
GurujiBot/1.0(+http:/ / www.guruji.com/ en/ WebmasterFAQ.html) |
1 |
0.00% |
0 |
418 |
DomainCrawler/2.0(info@domaincrawler.com; http:/ / www.domaincrawler.com/ therightstorage.com |
1 |
0.00% |
0 |
419 |
Mozilla/5.0(Macintosh;U;IntelMacOSX;en-US ;rv:1.8.1.14)Gecko/ 20080404Googlebot2.1 |
1 |
0.00% |
1 |
420 |
Mozilla/5.0(Windows;U;WindowsNT5.1;en-US; rv:1.9.2)Gecko/ 20100115Googlebot2.1 |
1 |
0.00% |
0 |
421 |
Dalvik/0.8.15(Prime;U;SunOS2.4.30;HAL9000 ;rv5.0)Trident/ 20030809Thunderspider/ 8.0.1 |
1 |
0.00% |
1 |
422 |
yacybot+(freeworld/global;+amd64+Linux+3. 2.0-4-amd64;+java+1.7.0_25;+Etc/ de)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
423 |
Dalvik/0.8.15(iPod;U;SymbOS1.0;M68100;rv1 .6.11)Gangsta/ 20090530Seaspider/ 5.0.1 |
1 |
0.00% |
0 |
424 |
INTERNETRADIOcrawler(+http:/ / www.radio.gvcbot.com:83) |
1 |
0.00% |
0 |
425 |
wf_crawler+(http://www.websitefigures.com );c2VvLWNyYXdsZXI= |
1 |
0.00% |
0 |
426 |
Godzilla/9.0(iPod;U;WindoofMobile2.2.18;A E32K;rv1.7.3)Gangsta/ 20040411Waterspider/ 2.0.0.20 |
1 |
0.00% |
0 |
427 |
Mozilla/5.0(Windows;U;WindowsNT5.1;en-US; rv:1.8.1.12)Gecko/ 20080201Firefox/ 2.0.0.9MEGAUPLOAD2.0Googlebot1988... |
1 |
0.00% |
0 |
428 |
FASTEnterpriseCrawlerusedbyILU(http:/ / www.ilu.co.jp/ ) |
1 |
0.00% |
0 |
429 |
Mozilla/5.0+(compatible;++3439;+jFirewall Bot/ 4.0.80;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
430 |
Tweetbot/13200+CFNetwork/ 596.4.3+Darwin/ 12.4.0+(x86_64)+(MacBookPro10%2C1) |
1 |
0.00% |
1 |
431 |
TheLemurWebCrawler/Nutch-1.3(LemurWebCraw lerusingNutch1.3;http:/ / boston.lti.cs.cmu.edu/ crawler_12/ ;admin@lemu... |
1 |
0.00% |
0 |
432 |
Mozilla/5.0+(compatible;+Shareaholicbot/ 1.0;++http:/ / www.shareaholic.com/ bot) |
1 |
0.00% |
1 |
433 |
BaiduImagespider+(+http:/ / www.baidu.jp/ search/ s308.html) |
1 |
0.00% |
0 |
434 |
Mozilla/5.0+(Linux;+Android+4.2.1;+CUBOT+ A6589S+Build/ JOP40D)+AppleWebKit/ 537.36+(KHTML,+like+Gecko)+Chrome/ 29.0.1547.72+Mobile+Safari/ 537.36+OPR/ 16.0.1212.65583 |
1 |
0.00% |
1 |
435 |
Tweetbot/13300+CFNetwork/ 673.0.2+Darwin/ 13.0.0+(x86_64)+(MacBookPro10%2C1) |
1 |
0.00% |
1 |
436 |
Dalvik/0.8.15(LCARS;I;WindozeCE7.1;Alpha; rv1.6.12)Salamander/ 20110101Turbospider/ 11.0a5 |
1 |
0.00% |
0 |
437 |
Godzilla/9.0(Winblows;N;Unix2.2.24;H316;r v1.7.2)Penguin/ 20100101Icebot/ 1.0.5 |
1 |
0.00% |
0 |
438 |
Mozilla/5.0+(compatible;++7289;+jFirewall Bot/ 4.0.90;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
0 |
439 |
yacybot(freeworld/global;x86WindowsVista6 .0;java1.6.0_29;Europe/ de)http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
440 |
yacybot+(freeworld/global;+x86+Windows+7+ 6.1;+java+1.7.0_45;+America/ fr)+http:/ / yacy.net/ bot.html |
1 |
0.00% |
1 |
441 |
Fast/Alltheweb crawler |
1 |
0.00% |
1 |
442 |
load-time.com+Load+Time+Check.+For+more+i nfo+go+to+http:/ / www.load-time.com/ bot.html |
1 |
0.00% |
1 |
443 |
Godzilla/9.0(Linux;I;DR-DOS2.6.31;MIPS64; rv1.2.4)Alligator/ 20070518Sunspider/ 11.0a6 |
1 |
0.00% |
1 |
444 |
Automattic+Analytics+Crawler/ 0.2;+http:/ / wordpress.com/ crawler/ |
1 |
0.00% |
1 |
445 |
Motzilla/4.0(iPhone;N;Windoze2.4.10;MDMX; rv1.6.7)Lizard/ 20110101Windbot/ 3.5.13 |
1 |
0.00% |
0 |
446 |
Mozilla/5.0+(compatible;++3421;+jFirewall Bot/ 4.0.97;++http:/ / www.jfirewall.com/ bot.html) |
1 |
0.00% |
1 |
447 |
Mozilla/5.0+(compatible;+FriendFeedBot/ 0.1;++Http:/ / friendfeed.com/ about/ bot) |
1 |
0.00% |
1 |
448 |
GoogleBot/1.5+(Ubuntu+7.0;+en-US;) |
1 |
0.00% |
1 |
|
Subtotal |
25,689,497 |
100.00% |
354,664 |
|
Total |
25,689,497 |
100.00% |
354,664 |
