The "ROBOTS" tells the search engine spiders to follow links on your site and list other pages. And the "REVISIT-AFTER" tells the search engine when it should come back and revisit your site to update its database.
How many tags are there? i have no definite answer for this one, cause i have seen so many out there, for example, "Content-Language", "COPYRIGHT", "PUBLISHER", "PUBLISHER-EMAIL", "CREATION", "REVISON", "IDENTIFIER-URL", "EXPIRES", "DESCRIPTION", "RESOURCE-TYPE", "RATING", "DISTRIBUTION", "KEYWORD", etc..
some i have no idea what that mean, but here is a link maybe you can find something helpful
Not to mention "Generator", "Author" and "Keywords" (plural). Meddle not in the affairs of dragons,
For you are crunchy, and good with mustard.
Thanks PepperPepsi,
I appreciate the information!
Thanks to you too tsdragon!
The ROBOTS META tag will be useful.
Do I need to specify which pages on your website the ROBOT tag has to find? Or will it just list all of the pages on your site by itself?
I was reading about the “ROBOTS” meta tag and I have a question.
I read on the link above…
“The INDEX directive specifies if an indexing robot should index the page.”
“The FOLLOW directive specifies if a robot is to follow links on the page.”
I understand the INDEX definition, but I don’t get what the FOLLOW directive does?
What’s the point of the FOLLOW directive following links on a page when the INDEX directive lists each page on your website in its search engine?
Use All or "INDEX, FOLLOW" if you want
-------------------------------------------------------
rebots to include the page in search services and follow links from this page to find other pages.
Use "INDEX, NOFOLLOW" if you want
-------------------------------------------------------
the page to be indexed, but do not want the rebot to explore the links from the page
Use All or "NOINDEX, FOLLOW" if you want
-------------------------------------------------------
the subsidiary links to be explored but do not want to indexed them
Use "NONE" if you want
-------------------------------------------------------
robot to ignore the page
hope this helps, Chiu Chan
cchan@emagine-solutions.com
Well, INDEX means wether or not the page (1 page, not the entire site) should be indexed so if i have a webpage of seventy pages and only one contains index and the rest is all noindex then only the page which contains index will show up when people search for your page.
For the FOLLOW bit, say that you have a page on your site that has for example 30 links. If you use FOLLOW on that page it will follow those links as well without even checking wether or not they belong to the same site. So if you set it to NOFOLLOW all those links will be skipped.
This way you can actually DEFINE what the robot finds and shows!!!
NOINDEX is handy for sites that use frames, because then you can make sure that only the first frame (if your site has multiple) is loaded.
NONE would be usefull for sites like my own (which btw shouldnt be visited with Netscape because then it looks like crap but im still working on that ...), which contain both a lot of links and (multiple) frames. This way I make sure that only the main page is indexed.
If you by the way want to use INDEX,FOLLOW then just leave the robots meta tag out because this is the default setting
(INDEX turned on and FOLLOW turned on).
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.