User-agent: * # This is a list of sub-directories which we do not want search engines to look at # edit this list where necessary #Note: the “Disallow: /maint/” and “Disallow: /admin/” have been left out because they are not typically linked # from the main site so therefore remain obscure. Putting them in the robots.txt file will tell anyone what # subdirectories you have. #Note 2: the Disallow: /swf/ directory has been left out because newer searchengines can now read Flash content. #Note 3: Remove any of the directories below which do not exist in a specific website. Disallow: /_notes/ Disallow: /cgi-bin/ Disallow: /common/ Disallow: /connections/ Disallow: /css/ Disallow: /images/ Disallow: /templates/ Disallow: /include/