CFHTTP Problem

[Decus]

I'm fairly new to ColdFusion and especially using the CFHTTP tag. Here's the problem:

We have a database storing over 3000 links (and increasing) to various web sites. What we want to do is create an automatic link checker that will probe each link stored in the database to test whether they're dead or not and then create an automatic report on the status of these links, the sections they belong to etc etc, to be stored in our database. It is for these requirements why we have not chosen to use third party link checking software.

The CF script I've written up till now in order to deal with the problem is:

<cfquery name=&quot;getlinks&quot; datasource=&quot;linksdatabase&quot; dbtype=&quot;ODBC&quot;>
Select url
From links
</cfquery>

<CFSET deadlinks = &quot;&quot;>
<cfoutput query=&quot;getlinks&quot;>
<CFSET linkaddress = getlinks.url>
<CFTRY>
<CFHTTP
URL = #linkaddress#
resolveurl = 0
throwonerror = Yes
TIMEOUT=10
>
</CFHTTP>

<CFCATCH TYPE=&quot;COM.ALLAIRE.COLDFUSION.HTTPFAILURE&quot;>
<CFSET deadlinks = deadlinks & #linkaddress#>
</CFCATCH>

<CFCATCH TYPE=&quot;COM.ALLAIRE.COLDFUSION.HTTPNOTFOUND&quot;>
<CFSET deadlinks = deadlinks & #linkaddress#>
</CFCATCH>

<CFCATCH TYPE=&quot;COM.ALLAIRE.COLDFUSION.HTTPAUTHFAILURE&quot;>
<CFSET deadlinks = deadlinks & #linkaddress#>
</CFCATCH>

<CFCATCH TYPE=&quot;any&quot;>
CFCATCH.Message #CFCATCH.Message# <br>
Exception Type: #CFCATCH.TYPE#
</CFCATCH>
</CFTRY>
</cfoutput>


<cfoutput><br><br>deadlinks = #deadlinks#</cfoutput>

However, when I run the above script, the process runs extremely slowly and eventually I get the message:

-------------------------------------
Error Diagnostic Information
unknown exception condition

unknown error while executing a tag.
-------------------------------------

Does anybody know why this problem is occuring? Is it because the script is very resource intensive and if so, is there a more efficient method I could use and/ or is there a way to manage memory in CF?