Hello,
I have rh7.1 installed hosting two of my domains. I have been trying to get one of my sites listed in google. On one of the google forums someone mentioned to look for googlebot in logs. The only thing I have seen resembling a search engine have been request for robots.txt but never see anything like googlebot or names of any other robots that have visited my server. I know googlebot has visited my server because one of my sites is listed with google. Is there something I need to configure to capture the names of search engine robots? My access log only shows ip addresses, time of request and page requested.
Thanks
I have rh7.1 installed hosting two of my domains. I have been trying to get one of my sites listed in google. On one of the google forums someone mentioned to look for googlebot in logs. The only thing I have seen resembling a search engine have been request for robots.txt but never see anything like googlebot or names of any other robots that have visited my server. I know googlebot has visited my server because one of my sites is listed with google. Is there something I need to configure to capture the names of search engine robots? My access log only shows ip addresses, time of request and page requested.
Thanks