User-agent: msnbot Crawl-delay: 10 Disallow: /movie/ Disallow: /Movie/ USer-agent: Bingbot Crawl-delay: 10 Disallow: /movie/ Disallow: /Movie/ User-agent: naverbot Disallow: / User-agent: ia_archiver Crawl-delay: 60 User-agent: yandex Disallow: / User-agent: yanga Disallow: / User-agent: TheRarestParser Disallow: / User-agent: SeznamBot Disallow: / User-agent: Robozilla Disallow: / User-agent: psbot Disallow: / User-agent: ccbot Disallow: / User-agent: OmniExplorer_Bot Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-agent: szukacz disallow: / User-agent: Whizbang Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Charlotte Disallow: / User-agent: MLBot Disallow: / User-agent:JadynAve Disallow: / User-agent: discobot Disallow: / User-agent: NetResearchServer Disallow: / User-agent: twiceler Disallow: / User-agent: dotbot Disallow: / User-agent: * Disallow: /Categories/ Disallow: /categories/ Disallow: /Weather/ Disallow: /weather/ Disallow: /Admin/ Disallow: /admin/ Disallow: /Merchant/ Disallow: /merchant/ Disallow: /Security/ Disallow: /security/ Disallow: /Search/ Disallow: /search/ Disallow: /Naughty/ Disallow: /naughty/ Disallow: /mail.asp Disallow: /Review/AddReview.aspx Disallow: /movie/ Disallow: /Movie/ Disallow: /redirect.asp Disallow: /Email.aspx Disallow: /Map.aspx