Is this a search engine robot?

[audiopro]

This may not be the correct forum but I am not sure what section this comes under. We have a section of our site which includes an example of an interactive calendar. Every minute of each day, this calendar is accessed by the bot detailed below. Not a problem except for giving misleading info in my log file, but why is this happening?
I have X'd out the remote address but it is always 1 of 4 different addresses.

$ENV{'REMOTE_ADDR'} - XX.XXX.XX.XXX
$ENV{'HTTP_USER_AGENT'} - Mozilla/5.0 (compatible; Googlebot/2.1; 
$ENV{'HTTP_REFERER'} - +[URL unfurl="true"]http://www.google.com/bot.html)[/URL] -

Keith

http://www.studiosoft.co.uk

 

[traingamer]

See: Google - remove pages from crawl


Greg
"Personally, I am always ready to learn, although I do not always like being taught." - Winston Churchill

 

[audiopro]

Thanks Greg
I want the page to be indexed as it is a specific aspect to our work which we do a lot of. It sems strange that this page is indexed every minute of every day but the rest of the site is indexed in the nornal way.

Keith

http://www.studiosoft.co.uk

 

[feherke]

Hi

Then better use a Google sitemap where you can specify how often you estimate that page's content is updated and Googlebot should visit it.

Feherke.

http://rootshell.be/~feherke/

 

[ChrisHirst]

I doubt that using a Google sitemap will make any real difference whatsoever to the crawl rate.
The issue is a prime example of how links affect the crawl rate of some internet search engines. Quite likely you will find that the msnbot is also a frequent visitor.

I have encountered the same issue on two sites, and in both instances a calendar was the point of interest. Even the empty entries were getting indexed on a hourly basis in some cases.
The reason is that most calendars generate a very well linked navigation system, thus giving many paths for the crawlers to be sent to the pages.

One solution is to block crawlers from the calendar system entirely and give them a seperate view of the pages with the content you want indexed then provide a link on these pages for "real" visitors to access the main calendar sections.

Chris.

Indifference will be the downfall of mankind, but who cares?
Woo Hoo! the cobblers kids get new shoes.
People Counting Systems

So long, and thanks for all the fish.

 

[audiopro]

Thanks Chris
I am a little reluctant to prevent any visits from indexing robots. Their visits continue although we are down to only a few stabs at the calendar. If search engine ranking depended on the number of robot visits, I should be up there above the sponsored links (lol).

Keith

http://www.studiosoft.co.uk