Is there a way.....

[fenris]

Is there a way to dupplicate windows ability to copy multiply files at the same time (multithread) using an api or something similar. I have been trying to get VB to do this with no luck. The reason that I am interested in this is that a project I am writing involves the transfer of several hundred files of 4Mb in size. I have discovered that multithreading (through java) can make the file tranfer much faster...

If anybody has any ideas or comments they would be appreciated.

Troy Williams B.Eng.
fenris@hotmail.com

 

[VB400]

If your talking about copying files from one folder to another you can use FileSystemObject. Make a reference to "Microsoft Scripting Runtime" then use the following code:

Dim fso As FileSystemObject

Set fso = New FileSystemObject
fso.CopyFile source, Destination, Overwrite

You can use multiples for the source such as "c:\*.dat"

I hope this helps!
Tarek

The more I learn, the more I need to learn!

 

[swilliams]

have you considered using the winsock control to ftp the files??
Simon

 

[fenris]

Simon, ftp the files? Would that work from across the network? That sounds like an interesting idea!

Tarek, would the system object allow me to have two or more files copying at the same time?

This is the scenerio, I have folder x located on a drive somewhere (either local or a mapped network drive). Folder x has 40,000+ files in it. I wish to copy out a few hundred of these files to folder y (again on either local or a mapped network drive). What I want to be able to do is have a couple of objects or controls, say obj1 and obj2 and they both have a fileDone event along with error checking events.

I would like to start the copy process by:
obj1.copy source, destination
obj2.copy source, destination

and when obj1 and obj2 raise the filedone event, they will be assigned another file to copy. This will proceed till the que is empty.


Thanks for the ideas.....



Troy Williams B.Eng.
fenris@hotmail.com

 

[fenris]

I tried using the filesystemobject, but it just ends up copying the files serially, that is, it waits till the first fso is done, then it starts the second one.

As far as the winsock control goes, that is my ultimate goal, to make a control that copies files and behaves simalar to that of the winsock control.

I even tried the copyfile api but it worked the same as the fso. I am wondering, do I have to put this stuff into a control and try it out on a test project or can I just put it into a class.



Troy Williams B.Eng.
fenris@hotmail.com

 

[SirLee]

Why do you want to copy the files all at the same time? - It will take much more time then copying them one by one.

This is a harddisk problem. The head has to reposition itself a lot two write in two files simultaneously.

Try it out in explorer and you will see what I mean.

 

[Alt255]

SirLee makes a valid point. It is impossible for an ordinary hard drive to write to more than one sector at any given time. If somebody has created a hard drive with multi-dimensional capabilities, e-mail me with the details ASAP. I want to invest in the technology.

The bottom line is hardware. Software trickery cannot make the hardware do things it wasn't designed to do.

You can stuff a cache until it is full but every item in the cache has to wait its turn. Welcome to the real world. The OS can *MULTI* whatever it pleases but the hardware must deal with the data one bit at a time.

In the end, you may find a faster way to stuff the cache, but you will not find a faster way to write data to disk.

Alt255@Vorpalcom.Intranets.com

"If you can get people to ask the wrong questions, they'll never find the right answers."[tt]
Thomas Pynchon[/tt]

Perhaps the reverse is also true....
​

 

[fenris]

I guess I should rephrase my question to pseudo multithreading!

I know I can copy more then one file at the same time and that it is quicker then copying the files serially. I wrote a program in java that uses threads to copy files. I used this program against another program that I had written in java that only copies files serially.

Coying the same files, the threaded program took 1 minute less time to copy the same files (of various sizes and locations) then the serial program. This by no means is conclusive testing, but it was good enough for me.

I realize that there are hardware limitations! But when I can pseudo-multithread in one language, why can't I do it in another language.

I posted the source in the sun java forum, do a search on multi threaded file copy and it should turn up.

When the java program is broken down, in essence what it is doing is simple copying a piece of one file for a very short time frame, then it moves on to another file and copies some of it, then it moves back to the first file to sopy some more and so on and so fourth till the files are completely copied. To be honest with you I would think that this would be slower. But my tests have proved me wrong.


I have posted this question to other forms and usenet and still haven't gotten an answer as to whether visual basic can do this. If it can not handle this then I would stop wasting my time look for answers that are not there ;-)

Let me reiterate what I am looking for:

Problem: I have an extensive directory structure that is rather large (40+ GB) and I would like to copy x amount of files to other directories (x generally consists of files in the range of 4 to 5 MB and is usually 150+ files to transfer). I have no problems with copying the files serially, using api, filesystemobject, etc. But, mostly for my own information and it seems that it has not been done before in VB, I would like to write a VB program that can copy these files via multithreading.


I appreciate the responses and comments!



Troy Williams B.Eng.
fenris@hotmail.com

 

[fenris]

I even tried using these api calls to get the file copy function to be multithreaded, let me tell you they are nothing to play with!

'Creates a new thread
Private Declare Function CreateThread Lib "kernel32" (ByVal lpThreadAttributes As Any, ByVal dwStackSize As Long, ByVal lpStartAddress As Long, lpParameter As Any, ByVal dwCreationFlags As Long, lpThreadID As Long) As Long
'Terminates a thread
Private Declare Function TerminateThread Lib "kernel32" (ByVal hThread As Long, ByVal dwExitCode As Long) As Long
'Sets the priority of a thread
Private Declare Function SetThreadPriority Lib "kernel32" (ByVal hThread As Long, ByVal nPriority As Long) As Long
'Returns the proirity of a thread
Private Declare Function GetThreadPriority Lib "kernel32" (ByVal hThread As Long) As Long
'Enables a disabled Thread
Private Declare Function ResumeThread Lib "kernel32" (ByVal hThread As Long) As Long
'Disables a thread
Private Declare Function SuspendThread Lib "kernel32" (ByVal hThread As Long) As Long
'Returns the handle of the current thread
Private Declare Function GetCurrentThread Lib "kernel32" () As Long
'Returns the ID of the current thread
Private Declare Function GetCurrentThreadId Lib "kernel32" () As Long




Troy Williams B.Eng.
fenris@hotmail.com

 

[Alt255]

I remain an open-minded skeptic. This author claims that true multi-threading capability was lost with the advent of VB6. Basically, he says "you can't do it":

http://www.execpc.com/~jrbock/

This article is a bit more optimistic:

http://www.desaware.com/articles/threadingL3.htm

It would be interesting to know how this works out for you and whether or not you actually see an increase in performance.

Here are some related links:

http://support.microsoft.com/support/kb/articles/q198/6/07.asp

http://msdn.microsoft.com/library/techart/msdn_threads.htm

http://msdn.microsoft.com/library/d...on98/vbconprojectoptionsforcodecomponents.htm

http://msdn.microsoft.com/library/periodic/period97/periodic/msj/thread997.htm

http://msdn.microsoft.com/library/techart/msdn_deadlock.htm

http://msdn.microsoft.com/library/techart/msdn_threadli.htm

http://msdn.microsoft.com/library/techart/msdn_onthefly.htm

http://support.microsoft.com/support/kb/articles/Q174/8/04.asp

http://msdn.microsoft.com/library/backgrnd/html/msdn_rightvb.htm

Good luck!

Alt255@Vorpalcom.Intranets.com

"If you can get people to ask the wrong questions, they'll never find the right answers."[tt]
Thomas Pynchon[/tt]

Perhaps the reverse is also true....
​

 

[fenris]

Alt 255,
thanks for the links! I did a bit of investigating on my own and the results were not to promising. So, for the time being I will settle for serial file transfers....

I did however read that vb.net(7) will support threading which will be very similar to what java can do. So when vb.net is officially released I should be able to achieve my dream



Troy Williams B.Eng.
fenris@hotmail.com

 

[Alt255]

There is a discussion about acheiving true multi-threading by using PowerBasic in conjunction with VB. Maybe that will work for you until VBnet comes out.

Alt255@Vorpalcom.Intranets.com
​

"To run Visual Basic, you must have certain hardware and software installed on your computer.
• Any IBM-compatible machine with an 80286 processor or higher.
• One megabyte of memory."

Visual Basic Programmer's Guide

 

[Alt255]

Oops, I forgot to include the URL.

http://www.execpc.com/~jrbock/

Alt255@Vorpalcom.Intranets.com
​

"To run Visual Basic, you must have certain hardware and software installed on your computer.
• Any IBM-compatible machine with an 80286 processor or higher.
• One megabyte of memory."

Visual Basic Programmer's Guide

 

[fenris]

I have been thinking about this problem off and on for a while know and I think that I have achieved something. I created a standard exe that can copy files from one directory to another with all the bells and whistles like a progress bar and fancy error checking. The thought occured to me, why not try starting two copies of the program and get them to copy different folders at the same time. Low and behold this worked!!! It may not be the best way to achieve multithreaded file copying but it is good enough for me, for now......


If anybody is interested I can send you the source and maybe you can help me tweak it so that it works better.....



Troy Williams B.Eng.
fenris@hotmail.com

 

[chiph]

I did however read that vb.net(7) will support threading which will be very similar to what java can do. So when vb.net is officially released I should be able to achieve my dream

Yes, VB7 will support free-threading. But you should be aware that this feature is simply a bigger gun to shoot yourself in the foot with. Unless you rigorously protect your code with SyncLocks (also a new feature in VB7), you'll have a problem with two or more threads interfering with each other.

Imagine if you were to write a multi-threaded file mover. Everything works great, until one day a user does the following:
Thread 1: Move file C:\ABC.txt to C:\Foo\ABC.txt
Thread 2: Move file C:\Bar\ABC.txt to C:\ABC.txt

If thread 1 finishes first, everything goes as expected. If thread 2 finishes first, it might get a "file exists" error. Or the wrong ABC.txt might end up in directory Foo. And what's worse, the results vary depending on which thread gets the most CPU cycles, so you can't predict which case will occur.

Chip H.

 

[moonshadow]

You could do this in VB6 using ActiveX exe controls. Create an ActiveX exe which does the file copying for you. I suggest building in a timer control into it. When the ActiveX exe is told which file to copy, it sets the timer and returns immediately to the calling program. The calling program can then create further instances of the ActiveX and give them files to copy. When the timer reaches 0 (I suggest after 1 second), the ActiveX exe starts copying the file. When it finishes it raises an event back to the calling program to say it's finished. The ActiveX control, because it's an exe runs on it's own processor thread, and you can have several of them running at once.
I'm interested in this problem because I want to copy the same named file off 300 PCs onto one server - copying them Serially is VERY slow. But if I can multitask...