Preventing Search Engines from Accessing Certain Directories - robots.

[DoubleV]

we have a robots.txt file in the root directory of our nimlok.com web site to stop web searches from accessing sertain directories. I have checked with several references to make sure it is written correctly, and cannot find any problems with it.
Here it is:

User-agent: *
Disallow: /cgi-bin/
Disallow: /distributor/
Disallow: /ecommerce/
Disallow: /cad/
Disallow: /robots/
Disallow: /imanager/
Disallow: /images/
Disallow: /technical/
Disallow: /analog3.0/
Disallow: /new/

But if you go to Google and search for "Nimlok" a link to the distributor directory is listed as a sub-section of the first result. I am not sure if other directories also appear in search results - I did not see them on the first page, but chances are - they are displayed somewhere later in search results.
What's the deal with it? --------------------------------------------------
Goals are dreams with deadlines

 

[Wullie]

Hi,

Try removing all the trailing slashes from the directories so that you end up with the following.

User-agent: *
Disallow: /cgi-bin
Disallow: /distributor
Disallow: /ecommerce
Disallow: /cad
Disallow: /robots
Disallow: /imanager
Disallow: /images
Disallow: /technical
Disallow: /analog3.0
Disallow: /new

Hope this helps Wullie

http://www.survivorhelp.co.uk

http://www.freshlookdesign.co.uk