The robots attribute, supported by several major search engines, controls whether search engine spiders are allowed to index a page, or not, and whether they should follow links from a page, or not.

Wikipedia
Using the robots meta Tag - Google