User-agent: * # directed to all spiders Disallow: /cgi-bin/ Disallow: /~intranet~/ Disallow: /srchadm/ Disallow: /dbWeb/ Disallow: /samples/ Disallow: /javadocs/