# robots.txt file for users.comlab.ox.ac.uk # # This file indicates to compliant web robots/spiders which parts # of this web site should not be traversed # [see http://info.webcrawler.com/mak/projects/robots/norobots.html] # Keep robots out of /cgi-bin and Geraint's welsh rhyming dictionary User-agent: * Disallow: /cgi-bin/ Disallow: /geraint.jones/gwasg.aredig/odliadur/odl/