Send a 404 based on IP address

[enormousson]

We have an intranet site that has inadvertently been indexed on google. In order to remove it we need to get google to crawl the site again, and receive 404's to remove it from the index.
We cannot mask the site at the firewall level as it is on the same server as many public sites so at the moment the options I can think of are:


RewriteCond %{REMOTE_ADDR} !^150.130.100
RewriteRule ^(.*) - [G,L]

this sends a 410 - no good for google.

RewriteCond %{REMOTE_ADDR} !^150.130.100

http://www.website.com/thispagedoesnotexistsend404

[L,R=301]

this sends a redirect to a page that then sends a 404, not a 404 directly

Order deny,allow
Deny from all
Allow from 150.130.100

sends forbidden 403 - again no good for google.

where 150.130.100 is the IP that should have access to the site.

I feel I must be missing an obvious easy way to achieve this. Does anyone have any suggestions?

thanks

 

[feherke]

Hi

In order to remove it we need to get google to crawl the site again, and receive 404's to remove it from the index.

Robots are intelligent, is enough to ask them to not index that pages. Use a robots.txt file to achive that.

User-agent: *
Disallow: /intra/pages/

RewriteCond %{REMOTE_ADDR} !^150.130.100

Note that the second parameter is a regular expression and period ( . ) is a metacharacter matching any character. So you have to escape them with a backslash ( \ ).

Feherke.

http://rootshell.be/~feherke/

 

[smah]

Also see thread65-1258146