Scanning files 2

[imosri]

Hi all

I got a request and I am starting to explore the options and feasibility.

The task is simple: To collect the number of files and the collective size of them for specific extensions on a computer.

Now you know were this request is headed. Looking for mp3, mpg, wma, jpg, etc. on the PCs.

The results will be then exported on an XML formated file to a known location and imported to a Database. Finally, the results are going to be compared against the PC inventory information that we already have.

Some requisites this app must observe:
1. Only has to scan local HDDs, no CD/DVD drive, no USB flash drives, no network drives, etc.
2. As silent and efficient as possible, no large HDD activity to show.

Being the scanning the key point of the application is the main question. Other issues like getting PC info (perhaps using WMI calls) and sending the resulting XML file to a Virtual directory (we know that the location is stored the Registry) are less important for now.

Thanks in advance for your response!!!

 

[strongm]

What ideas/suggestions have your IT team come up with?

 

[imosri]

Hi strongm,

My group is SQL/Asset Management/Compliance Managment oriented and our experince in VB is limited.

The main reason of my post is as the first paragraph states: Exploration and if it feasible.

Probably I didn't make the correct question.

Let me put it this way. I have been browsing this site and others and found some nice discussions and code examples about how to get file information and how to handle browsing thru subdirectories, etc..

This means the project is possible, I don't doubt we can get the files attributes, then sum up the file size and put the result on a XML file, move the resulting file and then do some SQL magic to load the info into a SQL table.

The question then is: In your experience, can we expect to have a fast scanning? I mean, if scanning a 80Gb HDD with, lets say, 10,000 target files is going to take 1 hour with the hard disk I/O going crazy, then the project is not feasable, because it fails to deliver requisite #2 of my original post.

For example, using my personal computer, using the simple XP Search, it took 10 minutes to find 3,000 mp3 files (all legal BTW ) on my 80Gb HDD, lot of HDD activity also, that's not good.

Someone may reply that there is some fancy call and may we can get a fast response time. We can then research, and make more intelligent questions if needed.

Thank you for your response.

Regards.

 

[Danp129]

I wouldn't expect anything faster than XP's search can scan. You can add a sleep or wait statement in the loop to lessen the I/O load, but it will make it take longer to scan of course.

You could also try Windows Desktop Search and query the index using ADO.

 

[harebrain]

Try this: open a command window on your computer and do:

C:
dir *.jpg /s

You can try any extensions you like. Chances are you're not going to get any faster than this. Also, watch the HDD activity light.

For that matter, you could roll everything you want to do up into a DOS batch file and capture the output, although that's not going to be in XML (which seems a little silly, anyway, especially if you want to be unobtrusive.)

 

[imosri]

Your post is an interesting idea. This brings the next ideas:

1.- Users are requested to lock their computers when they are away, locking is enforced also by policy after 15 minutes of inactivity.

It is possible to detect the lock state with VB?. If so, we can run our app and wait until there is "idle" time on the PC, kind of Seti@home.

2.- Another way to go is that the application only search the most probale folders to find the target files, there is a lot of .gifs and .wavs on the Windows folder, but those are ok, so we can save scanning 15K+ files on the Windows folder. BTW who store their music on the OS's folder?. Program Files folder will be trickier.

Harebrain,

The object of the application is to get statistics (not forensics) in order to enforce Compliance and Governance on the Organization's Assets. You know the drill, users using >30% of their computers for storing their iTunes Library, etc.

Using your suggestion on my personal PC, it found the same 3,000+ mp3s in 50 seconds, much, much better. To our use, from the result, only one row is useful

3801 File(s) 20191379350 bytes (using dir *.jpg /S /-C)

We can collect from every generated file that specific row.

We can try that...


Finally, XML formating of the results is just that, a format. It just that we have the experience to load a lot of XML files into SQL.

Regards

 

[harebrain]

DOS's DIR command produces statistics, so I don't understand the statement about forensics. And I do know the drill.

My statement about XML is that using it will leave a big footprint in your application, both in size and resource consumption; it's counterproductive to your goals. Surely you can parse "n files/m bytes" for SQL insertion without the XML middleman.

 

[imosri]

Hi harebrain,

I need just the number of files and the total size (statistics), not the individual filenames (evidence). Your DOS' Dir command gives both, we will just parse the info that we need.

We are not looking for who has the new Carmen Electra's picture!!!

About XML:
Probably I did not explain myself correctly. After collecting the info (parsing the Dir commands resulting files) we will put the results into a file, that output file (XML formatted) will be uploaded to a known location.

On the Server, we will have a process to read into SQL all the files, we have routines to perform that kind of tasks already.

The last action of our planned application is to upload a file to a known location, not to connect to SQL and insert the records.

Regards

 

[BobRodes]

<It is possible to detect the lock state with VB?. \

check this:

http://www.microsoft.com/technet/scriptcenter/resources/qanda/nov04/hey1115.mspx

 

[imosri]

Thanks BobRodes,

We do run a screen saver and lock the computer by using GP.

We are going to give it a try.

Thanks again.

Regards,

 

[BobRodes]

I would also suggest a deeper look into WMI. There is a great deal that you can do with it.