User-agent: 008 Disallow: / User-agent: Semrushbot-SA Crawl-delay: 10 User-agent: Semrushbot-SI Crawl-delay: 10 User-agent: Yahoo Pipes 2.0 Disallow: / User-agent: * Disallow: /admin/ Disallow: /secure/ Disallow: /administration/ Disallow: /wp-admin/ Disallow: /activate/ Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /~batman/ Disallow: /status/ Disallow: /remote.axd/ Disallow: /elmah.axd Disallow: /drupal/ Disallow: /umbraco/ Disallow: /*/shared-content/ Disallow: /*/search-results Disallow: /es/market-news-blog/forex-news/ User-agent: MJ12bot Crawl-delay: 119