abot/0.1 (abot; http://www.abot.com; abot@abot.com)
aipbot/1.0 (aipbot; http://www.aipbot.com; aipbot@aipbot.com)
amaya/8.3 libwww/5.4.0
Mozilla/5.0 (compatible; AnsearchBot/1.0; +http://www.ansearch.com.au/)
ArtfaceBot (compatible; MSIE 6.0; Mozilla/4.0; Windows NT 5.1;)
Mozilla/5.0 (compatible; BecomeBot/2.2.1; MSIE 6.0 compatible; +http://www.become.com/webmasters.html)
boitho.com-dc/0.70 ( http://www.boitho.com/dcbot.html )
boitho.com-dc/0.71 ( http://www.boitho.com/dcbot.html )
$BotName/$BotVersion [en] (UNIX; U)
CacheBot/0.445 (compatible, http://www.cachebot.com)
Mozilla/2.0 (compatible; MSIE 3.0; Update a; AK; Windows 95) via proxy gateway CERN-HTTPD/3.0 libwww/2.17
Clushbot/3.31-BinaryFury (+http://www.clush.com/bot.html)
ConveraMultiMediaCrawler/0.1 (+http://www.authoritativeweb.com/crawl)
CosmixCrawler/0.1
crawler (ISSpider-3.0; http://www.yoururlgoeshere.com)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; GoogleBot; Crazy Browser 1.0.5; .NET CLR 1.0.3705; .NET CLR 1.1.4322)
curl/7.10.2 (powerpc-apple-darwin7.0) libcurl/7.10.2 OpenSSL/0.9.7b zlib/1.1.4
curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6 OpenSSL/0.9.7a ipv6 zlib/1.1.4
curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6 OpenSSL/0.9.7a ipv6 zlib/1.2.0.7
curl/7.11.2 (i686-pc-linux-gnu) libcurl/7.10.2 OpenSSL/0.9.6i ipv6 zlib/1.1.4
curl/7.12.0 (i686-pc-linux-gnu) libcurl/7.12.0 OpenSSL/0.9.7e ipv6 zlib/1.2.2
curl/7.7.2 (powerpc-apple-darwin6.0) libcurl 7.7.2 (OpenSSL 0.9.6b)
curl/7.7.3 (i686-pc-linux-gnu) libcurl 7.7.3 (OpenSSL 0.9.6)
curl/7.7.3 (win32) libcurl 7.7.3 (OpenSSL 0.9.6)
curl/7.9.3 (powerpc-ibm-aix4.3.3.0) libcurl 7.9.3 (OpenSSL 0.9.6m)
curl/7.9.5 (i386-redhat-linux-gnu) libcurl 7.9.5 (OpenSSL 0.9.6b) (ipv6 enabled)
curl/7.9.8 (i386-redhat-linux-gnu) libcurl 7.9.8 (OpenSSL 0.9.7a) (ipv6 enabled)
dloader(NaverRobot)/1.1
EasyDL/3.04  http://keywen.com/Encyclopedia/Bot
googlebot/2.1 (+http://www.googlebot.com/bot.html
googlebot/2.1; +http://www.google.com/bot.html
Googlebot (+http://www.google.com/bot.html)
Googlebot/2.1 (compatible; MSIE; Windows)
Googlebot/2.1 (+http://www.googlebot.com/bot.html) (compatible; MSIE 6.0; )
Googlebot/2.1+(+http://www.google.com/bot.html)
Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 FAKE (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.)
GoogleBot/2.1
Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html
grub crawler
grub crawler(http://www.grub.org)
gsa-crawler (Enterprise; GID-01742;gsatesting@rediffmail.com)
HooWWWer/2.1.0 (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-info<at>hiit.fi)
Hyperlink/2.5e (Commodore 64)
Java/1.4.1_01
Java/1.4.1_02
Java/1.4.1_04
Java/1.4.1_05
Java/1.4.2_01
Java/1.4.2_03
Java/1.4.2_04
Java/1.4.2_05
Java/1.4.2_06
Java/1.5.0_01
Java/1.5.0-beta2
Mozilla/4.0 (compatible; Kittiecentral.com; Index-Client)
Kurzor/1.0 (Kurzor; http://adcenter.hu/docs/en/bot.html; cursor@easymail.hu)
LB-Crawler/1.0
libwww-perl/5.79
SonyEricssonT306/R101 [Html2Wml/0.4.11 libwww-perl/5.79]
testJapanequeDelicious/0.1 libwww-perl/5.803
W3C-checklink/4.1 [4.14] libwww-perl/5.803
lmspider (lmspider@scansoft.com)
Mozdex/0.06-dev (Mozdex; http://www.mozdex.com/bot.html; spider@mozdex.com)
Mozilla/3.04 (X11; I; SunOS 5.5.1 sun4u) via HTTP/1.0 depraved-midget.onlinescanner.com/
Mozilla/4.0 (compatible; IE-Favorites-Check-0.5)
Mozilla/4.0 (SKIZZLE! Distributed Internet Spider v1.0 - www.SKIZZLE.com)
Mozilla/4.0 (stat 0.12) (statbot@gmail.com)
Mozilla/5.0 (Clustered-Search-Bot/1.0; support@clush.com; http://www.clush.com/)
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;) Unchaos/Crawler
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20041027 NaverBot/0.9.3
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 GoogleBot/1.1
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040730 Googlebot/2.1/2.1
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040707 Lightningspider/0.9.2
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040803 Superlizard/0.9.3
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7) Gecko/20040805 Googlebot/2.1
Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot BETA 1.2 (http://www.voila.com/)
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; T-Online Internatinal AG; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; DT; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; QXW0338t; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; FunWebProducts; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; .NET CLR 1.0.3705; Googlebot; .NET CLR 1.1.4322; .NET CLR 1.0.3705; Googlebot)
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; T312461; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows compatible LesnikBot)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; 3COM U.S. Robotics)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Girafabot; girafabot at girafa dot com; http://www.girafa.com)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; BOTW)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Googlebot)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; http://www.pregnancycrawler.com)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; AskBar 3.00; .NET CLR 1.1.4322; Fluffi Bot+)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; CDSource=v9e.03; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts-MyWay; .NET CLR 1.0.3705; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Googlebot)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Googlebot/2.1 (+http://www.googlebot.com/bot.html))
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Googlebot/2.1 (+http://www.googlebot.com/bot.html); Maxthon; FDM)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; Googlebot; .NET CLR 1.1.4322)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Roadrunner; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SC/5.60/1.01/FS-Internett; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; stokeybot)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; DigExt; FunWebProducts; Media Center PC 3.0; .NET CLR 1.0.3705; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 3.1; Googlebot/2.1)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.40607; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322; MSIECrawler)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT; MS Search 4.0 Robot)
Mozilla/4.0 (compatible; MSIE 6.0; Windows XP Professional Bot v.5.)
Mozilla/3.0 (compatible; MuscatFerret/1.6.3; claude@euroferret.com) via proxy gateway CERN-HTTPD/3.0 libwww/2.17
NP/0.1 (NP; http://www.nameprotect.com; npbot@nameprotect.com)
NutchCVS/0.01-beta (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
NutchCVS/0.03-dev (Nutch; http://www.nutch.org/docs/bot.html; nutch-agent@lists.sourceforge.net)
NutchCVS/0.03-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
NutchCVS/0.06-dev (bot; http://www.nebel.de:8080/en/bot.html; nutch-agent@nebel.de)
NutchCVS/0.06-dev (bot; http://www.nebel.de/bot.html; nutch-agent@nebel.de)
NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; jagdeepssandhu@hotmail.com)
NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
NutchCVS/0.06-dev (Nutch; http://www.s14g.com/en/search.html; travelbot@s14g.com)
NutchOSUOSL/0.05-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
NWSpider 0.9
ObjectsSearch/0.01 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com)
ObjectsSearch/0.06 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com)
ObjectsSearch/0.07 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com)
Opera/7.60 preview 4 (Mac OS X 10.3)
Opera 7.60 (X11; Linux i386; U) [en] via HTTP/1.0 evil-uncle.trojanchecker.com/
PhpDig/PHPDIG_VERSION (+http://www.phpdig.net/robot.php)
pipeLiner/0.10 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html)
POE-Component-Client-HTTP/0.510 (perl; N; POE; en; rv:0.510)
Python-urllib/1.15
Python-urllib/2.0a1
Python-urllib/2.0a1, Dowser/0.26 (http://dowser.sourceforge.net)
SafariBookmarkChecker/1.27 (+http://www.coriolis.ch/)
savvybot/0.2
ShowLinks/1.0 libwww/5.4.0
Simpy/1.1 (Simpy; http://www.simpy.com/?ref=bot; feedback at simpy dot com)
SpiderMan 3.0.1-2-11-111 (CP/M;8-bit)
Steeler/2.0 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/)
Tutorial Crawler 1.4
VM4050/132.037 UP.Browser/6.2.2.4.e.1.100 (GUI) MMP/2.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
WebZIP/4.1 (http://www.spidersoft.com)
Wget/1.8.2 modified
Wget/1.9+cvs-stable (Red Hat modified)
WordPress/1.2.1 PHP/4.3.9-1
Wotbox/0.7-alpha (bot@wotbox.com; http://www.wotbox.com)
Wotbox/alpha0.6 (bot@wotbox.com; http://www.wotbox.com)
Yahoo-MMAudVid/1.0 (mms dash mmaudvidcrawler dash support at yahoo dash inc dot com)
YottaShopping_Bot/4.12 (+http://www.yottashopping.com) Shopping Search Engine
