About “robots.txt“.  When you got a folder or a complete site even that you don’t want to be indexed and searchable on on Google or Bing, you can easily do this by creating a robots.txt file and put it at the top (root) folder of your site. When a robot enters your site it will first of all read this simple text file and index only what you want it to index. Though a warning, you have to keep in mind, that this will only be respected by good robots. There are people making their own robots to find mails, images, addresses or whatever that would not even read it. Anyway, some robot stuff: Disallow ALL:

User-agent: *
Disallow: /

Allow ALL:

User-agent: *
Disallow:

Exclude some folders for ALL robots:

User-agent: *
Disallow: /documents.html
Disallow: /misc.php
Disallow: /cgi-bin/ 
Disallow: /private/

Allow only Google:

User-agent: Googlebot
Disallow:

User-agent: *
Disallow: /

 Allow the bingbot access to the private section:

User-agent: *
Disallow: /private/

User-agent: bingbot
Disallow:

 Well, there you go, a few useful robots.txt examples… Cheers /jima