Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations gkittelson on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

question on Inktomi for submitting site map page as tool for spider...

Status
Not open for further replies.

spewn

Programmer
May 7, 2001
1,034
i heard if you submit a static html page with links to different pages within your site (like a site map) to a search engine, it will have a better chance of being spidered because it's a .html page and not a dynamically generated page.

ok. but what about pay search engines such as inktomi. if i submit a website to their search, that exact address appears on the search results for hotbot, overture, etc...

but what if i give them the address of the site map, they spider it? or will that actual site map address still show up as the result?

- g
 
For Inktomi paid inclusion (such as with PositionTech), if you pay for and submit one page for spidering/indexing then that is all that they are obligated to do.

They may elect to include other of your pages, but I've never heard of this happening. In fact, I've heard of the opposite. --------------
 
Hi - I got confirmation from Market Leap that if you pay for Inktomi to spider one page of your site, that spider will then only spider that page. It is best to let the spider find your site on its' own and pick up all the pages. Market Leap did say that the spider will not crawl through a dynamic site. I think that it is that the spider will not crawl through complex urls, and a simple page with a .asp extension should be fine as they are crawling through my site approx. every 2 days and my home page has a .asp extension.

The Inktomi search technology was recently revamped, a years worth of development, and the spider now will crawl through your webpages content and it no longer supports the Meta Keywords tag.

-Beth beth@integratedresourcemgmt.com
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top