User-agent: * Disallow: /cgi-bin/ Disallow: /incl/ Disallow: /admin/ Disallow: /search/ Disallow: /gb/ Disallow: /kcaptcha/ Disallow: /main/ Dissalow: /subscr/ Dissalow: /community/ Dissalow: /creation/ Dissalow: error* Request-rate: 1/5 # maximum rate is one page every 5 seconds Visit-time: 0200-0800 # only visit between 06:00 and 08:45 UTC (GMT) User-agent: ia_archiver User-agent: Googlebot Disallow: /cgi-bin Disallow: /search Disallow: /gb Request-rate: 1/5 # maximum rate is one page every 5 seconds Visit-time: 0200-0800 # only visit between 06:00 and 08:45 UTC (GMT)