Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations gkittelson on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

robots.txt for the whole site... 1

Status
Not open for further replies.

Extras

Technical User
Nov 16, 2001
232
US
i am setting up a test site and do not want it crawled by search engines.

I was wondering if the following 2-line code within the robots.txt will prevent any search engine from crawling the root and any other directories within the site. i.e. I do not want any part of the site crawled...

TIA!



User-agent: *
Disallow: /
 
A simple but grateful - THANKS!
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top