# don't index cgi stuff User-agent: * Disallow: /cgi-bin Disallow: /fcgi-bin Disallow: /db/state/maintainers Disallow: /ripe/mail-archives/uknot Disallow: /perl/whois Disallow: /images Disallow: /pics Disallow: /membership/members/indices Disallow: /rs/ipv6/allocs6.html Disallow: /includes Disallow: /templates Disallow: /projects/ttm/cgi-bin Disallow: /projects/ttm/Plots Disallow: /error Disallow: /sitemap.html Disallow: /info/stats/www2 Disallow: /info/stats/www3 Disallow: /countries Disallow: /search Disallow: /ssi Disallow: /css Disallow: /membership/new-members/survey Disallow: /statistics/site Disallow: /statistics/www3 Disallow: /info/stats/www3 Disallow: /info/stats/site Disallow: /dfk Disallow: /info/ncc/pfund Disallow: /ripe/wg/draft-minutes Disallow: /meetings/regional/qatar-2006/test User-Agent: W3C-checklink Disallow: