User-agent: * Disallow: /downloads/ Disallow: /downloads/admin/ Disallow: /cgi-bin/ Disallow: /guestbook/ Disallow: /error/ User-agent: grub-client Disallow: / User-agent: ZyBorg/1.0 Disallow: / User-agent: ZyBorg/1.0 (wn.zyborg@looksmart.net; http://www.WISEnutbot.com) Disallow: / User-agent: ZyBorg/1.0 Dead Link Checker (wn.zyborg@looksmart.net; http://www.WISEnutbot.com) Disallow: / User-agent: Mediapartners-Google* Disallow: User-agent: libwww-perl/5.65 Disallow: / User-agent: TurnitinBot Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-agent: proszynski.pl Disallow: / User-agent: Microsoft.URL Disallow: / User-agent: msnbot/0.11 (+http://search.msn.com/msnbot.htm) Disallow: / User-agent: Wget Disallow: / User-agent: Lynx Disallow: / User-agent: Curl Disallow: / User-agent: dloader(NaverRobot)/1.0 Disallow: / User-agent: psbot Disallow: / User-agent: Baiduspider Disallow: /