############################### # # User-agent: * # # list folders robots are not allowed to index # #Disallow: /About/ #Disallow: /aspnet_client/ #Disallow: /careers/ #Disallow: /Crohn/ #Disallow: /Epub/ #Disallow: /images/ # # list specific files robots are not allowed to index # # # ###############################