Hi,
I would like to index a website that is a subdir on the root domain (
The problem is that pages on the site I'm indexing contains links back to the root domain, so pages at other subdirs of the domain are indexed and included in the crawl.
I have the following set of rules when this happens:
Included (complex urls enabled)
Included
If I add the line
Exclude it overrides the two previous rules.
How can I make it exclude ALL dirs on the root domain even if there are links to those dirs from the documents in the dir I WISH to index.
Thanks in advance for any tips regarding this issue.
Sean.
I would like to index a website that is a subdir on the root domain (
The problem is that pages on the site I'm indexing contains links back to the root domain, so pages at other subdirs of the domain are indexed and included in the crawl.
I have the following set of rules when this happens:
Included (complex urls enabled)
Included
If I add the line
Exclude it overrides the two previous rules.
How can I make it exclude ALL dirs on the root domain even if there are links to those dirs from the documents in the dir I WISH to index.
Thanks in advance for any tips regarding this issue.
Sean.