Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Register Log in
  • Congratulations gkittelson on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Robots Text

Status
Not open for further replies.
spartiums

spartiums

Technical User
Feb 22, 2005
44
GB
Do you need Robots text on your site - will my site be spidered and indexed with the search engines if I don't include it?

Thanks, Tina
 
Sort by date Sort by votes
If you haven't got a robots.txt file then bots will still index your site.
 
Upvote 0 Downvote
robots.txt is an exclusion method you only need it to tell the bots that obey it NOT to index something.

Though having a blank one will cut down the error log entries.


Chris.

Indifference will be the downfall of mankind, but who cares?
Woo Hoo! the cobblers kids get new shoes.
People Counting Systems

So long, and thanks for all the fish.
 
Upvote 0 Downvote
Status
Not open for further replies.

Similar threads

HankReadon
  • Locked
  • Question
Graphics co users to input text, upload JPEGs and preview their signs
Replies
0
Views
45
HankReadon
HankReadon
melodyr
  • Locked
  • Question
Change COD text
Replies
0
Views
40
melodyr
melodyr
kainickara
  • Locked
  • Question
Gentran 850 Mapping - MSG segment - how to split text
Replies
0
Views
33
kainickara
kainickara
spewn
  • Locked
  • Question
is the robots meta tag a guarantee that the page will/won't be indexed
Replies
2
Views
31
johnwm
johnwm
jstreich
  • Locked
  • Question
Text and background
Replies
9
Views
61
jstreich
jstreich

Part and Inventory Search

Sponsor

Back
Top