# Alle Roboter einladen User-agent: * Disallow # Alle Roboter von gewissen Verzeichnissen fernhalten User-Agent: * Disallow: /admin/ Disallow: /cgi-bin/ Disallow: /attachments/ Disallow: /images/ Disallow: /templates/ Disallow: /templatessection/ # Bestimmte Robotor einladen User-Agent: almaden.ibm.com Disallow: User-Agent: appie 1.1 Disallow: User-Agent: architext Disallow: User-Agent: ask jeeves Disallow: User-Agent: asterias2.0 Disallow: User-Agent: augurfind Disallow: User-Agent: baiduspider Disallow: User-Agent: bannana_bot Disallow: User-Agent: bdcindexer Disallow: User-Agent: crawler Disallow: User-Agent: crawler@fast Disallow: User-Agent: docomo Disallow: User-Agent: fast-webcrawler Disallow: User-Agent: fluffy the spider Disallow: User-Agent: frooglebot Disallow: User-Agent: geobot Disallow: User-Agent: googlebot Disallow: User-Agent: gulliver Disallow: User-Agent: henrythemiragorobot Disallow: User-Agent: ia_archiver Disallow: User-Agent: infoseek Disallow: User-Agent: kit_fireball Disallow: User-Agent: lachesis Disallow: User-Agent: lycos_spider Disallow: User-Agent: mantraagent Disallow: User-Agent: mercator Disallow: User-Agent: moget/1.0 Disallow: User-Agent: muscatferret Disallow: User-Agent: nationaldirectory-webspider Disallow: User-Agent: naverrobot Disallow: User-Agent: ncsa beta Disallow: User-Agent: netresearchserver Disallow: User-Agent: ng/1.0 Disallow: User-Agent: osis-project Disallow: User-Agent: polybot Disallow: User-Agent: pompos Disallow: User-Agent: scooter Disallow: User-Agent: seventwentyfour Disallow: User-Agent: sidewinder Disallow: User-Agent: sleek spider Disallow: User-Agent: slurp/si Disallow: User-Agent: slurp@inktomi.com Disallow: User-Agent: steeler/1.3 Disallow: User-Agent: szukacz Disallow: User-Agent: t-h-u-n-d-e-r-s-t-o-n-e Disallow: User-Agent: teoma Disallow: User-Agent: turnitinbot Disallow: User-Agent: ultraseek Disallow: User-Agent: vagabondo Disallow: User-Agent: voilabot Disallow: User-Agent: w3c_validator Disallow: User-Agent: zao/0 Disallow: User-Agent: zyborg/1.0 Disallow: