# exclude help system from robots User-agent: * Disallow: /manual/ Disallow: /doc/ Disallow: /gif/ Disallow: /cgi-bin/ Disallow: /www-bin/ Disallow: /bot-fodder/ # but allow htdig to index our doc-tree User-agent: susedig Disallow: