Hey all,
Sorry for the low points on this, kinda used most of them up. Hopefully someone will still have the answer.
Anyway im trying to run a recursive file scan of a file server. The Server has about 3TB of data so its pretty big. Anyway im using Windows API calls and using FindFirstFile and FindNextFile.
The scanner works perfectly on a small test folder, but when i get to a larger folder i run into issues.
The scanner will enter and leave directories recussively for a good while, scanning about 70,000 files and directories. However after a while it eventually dies. I added in some calls to GetLastError and some logging and found out that The function tried to make a call to GetFirstFile of a directory and returns an error code of 4. That equates out to being "Too many open files". Does anyone know how i can get around this problem?
My code is pasted below for the function having the issues, thanks.
bool FindFile::getFiles (HANDLE &searchHandle, WIN32_FIND_DATA &fileData,
string path)
{
int nValid;
if (searchHandle == NULL)
{
string pathToSearch = combinePath(path, "*");
searchHandle = FindFirstFile(pathToSearch.c_str(), &fileData);
if (searchHandle == INVALID_HANDLE_VALUE)
{
cout<<"FindFirstFile"<<endl;
cout<<GetLastError();
if (GetLastError() == ERROR_NO_MORE_FILES)
{
FindClose(searchHandle);
searchHandle = NULL;
}
}
nValid = (searchHandle == INVALID_HANDLE_VALUE) ? 0 : 1;
}
else
{
nValid = FindNextFile(searchHandle, &fileData);
if (nValid == 0) {
cout<<GetLastError();
if (GetLastError() == ERROR_NO_MORE_FILES)
{
FindClose(searchHandle);
searchHandle = NULL;
}
}
}
while (nValid)
{
// As long as this file is not . or .., we are done
if (strcmpi (fileData.cFileName, ".") != 0 &&
strcmpi (fileData.cFileName, "..") != 0)
return true;
nValid = FindNextFile(searchHandle, &fileData);
if (nValid == 0) {
cout<<GetLastError();
if (GetLastError() == ERROR_NO_MORE_FILES)
{
FindClose(searchHandle);
searchHandle = NULL;
}
}
}
FindClose(searchHandle);
searchHandle = NULL;
return false;
}
p.s. if the code looks firmilar some of it (before my modifications) was taken from a program by Louka Dlagnekov that I found online. Just dont want people to think im claiming it as my own.
Sorry for the low points on this, kinda used most of them up. Hopefully someone will still have the answer.
Anyway im trying to run a recursive file scan of a file server. The Server has about 3TB of data so its pretty big. Anyway im using Windows API calls and using FindFirstFile and FindNextFile.
The scanner works perfectly on a small test folder, but when i get to a larger folder i run into issues.
The scanner will enter and leave directories recussively for a good while, scanning about 70,000 files and directories. However after a while it eventually dies. I added in some calls to GetLastError and some logging and found out that The function tried to make a call to GetFirstFile of a directory and returns an error code of 4. That equates out to being "Too many open files". Does anyone know how i can get around this problem?
My code is pasted below for the function having the issues, thanks.
bool FindFile::getFiles (HANDLE &searchHandle, WIN32_FIND_DATA &fileData,
string path)
{
int nValid;
if (searchHandle == NULL)
{
string pathToSearch = combinePath(path, "*");
searchHandle = FindFirstFile(pathToSearch.c_str(), &fileData);
if (searchHandle == INVALID_HANDLE_VALUE)
{
cout<<"FindFirstFile"<<endl;
cout<<GetLastError();
if (GetLastError() == ERROR_NO_MORE_FILES)
{
FindClose(searchHandle);
searchHandle = NULL;
}
}
nValid = (searchHandle == INVALID_HANDLE_VALUE) ? 0 : 1;
}
else
{
nValid = FindNextFile(searchHandle, &fileData);
if (nValid == 0) {
cout<<GetLastError();
if (GetLastError() == ERROR_NO_MORE_FILES)
{
FindClose(searchHandle);
searchHandle = NULL;
}
}
}
while (nValid)
{
// As long as this file is not . or .., we are done
if (strcmpi (fileData.cFileName, ".") != 0 &&
strcmpi (fileData.cFileName, "..") != 0)
return true;
nValid = FindNextFile(searchHandle, &fileData);
if (nValid == 0) {
cout<<GetLastError();
if (GetLastError() == ERROR_NO_MORE_FILES)
{
FindClose(searchHandle);
searchHandle = NULL;
}
}
}
FindClose(searchHandle);
searchHandle = NULL;
return false;
}
p.s. if the code looks firmilar some of it (before my modifications) was taken from a program by Louka Dlagnekov that I found online. Just dont want people to think im claiming it as my own.