threads and file copying

fenris · Jun 27, 2000

I am learning about threads, currently I can't see how it would be advantageous to use threads in copying multiple files from one location to another. I have a class that takes two arguments, the source file and the destination. This class works by doing a byte copy from the original to the copy. It works fine, but I am wondering if there is a way to get the O/S to do this. Is there a facility in java to allow for the creation of OS objects that would have some of the basic properties supported by the O/S yet still remain platform independent. As far as I know every O/S has facilities to move and copy files around. I know that one can use java's shell feature, but this would be platform specific. As far as Threads go how would I implement a simple class that uses, say two threads, too copy multiple files, say 100. The only thing I can think of is to pass the first 50 to one thread and the rest to the other one. If I am reading things correctly about threads, in order for one thread to be active, all the rest must be inactive. How would this apply to file I/O? puzzled, fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

carpeliam · Jun 27, 2000

well at the precise moment of action, only one thread can be run by the processor; but if all threads had to wait for others to finish before they could begin, that would defeat the purpose of threads. I think the only way that Java can perform platform-independent IO is through the standard classes you've already seen. Are there any Swing I/O classes? I still have not gotten a chance to even browse Swing documentation. Create a class that implements threads... this will do the I/O. You should be able to setDaemon(true), I think. From your main method outside of that class, have 2 arrays of filenames (or something similar, depending on your program demands); one for input source and another for output destination. For each item in the array, start a new thread with the specified source/destination info. You should be able to have 100 threads all doing I/O at the same time. If you run into problems, let me know

) Liam Morley <a href=mailto:lmorley@wpi.edu>lmorley@wpi.edu</a> <a href=

http://www.wpi.edu/~lmorley/>

] :: imotic :: website :: [</a> "light the deep, and bring silence to the world. 
light the world, and bring depth to the silence.

fenris · Jun 27, 2000

Thanks Liam, I just wanted to start with two threads, just to see how it worked. If I got that working then I would try more and see what kind of demands that placed on the system. Thanks fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

carpeliam · Jun 27, 2000

That would be slightly more difficult, but not impossible. Pass the entire array to both threads- when a thread processes a certain index for the array, flag that index somehow, so the other thread knows it's already been taken (this would have to be in a global variable). Then, keep processing until there's nothing left unflagged. That's better than splitting it up by number of files, since not every file will take the same amount of time. That's not the easiest thing to implement, but it's possible, and it sounds like a fun project. Liam Morley <a href=mailto:lmorley@wpi.edu>lmorley@wpi.edu</a> <a href=

http://www.wpi.edu/~lmorley/>

] :: imotic :: website :: [</a> "light the deep, and bring silence to the world. 
light the world, and bring depth to the silence.

fenris · Jun 28, 2000

Thanks for the information. I have been doing some reading on threads, and from what I understand if there are two many threads active, it can actually decrease performance. That is why I wanted to start with two threads, and go from there. Hopefully the class I create will allow an input for the number of threads involved in copying. fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

palbano · Jun 28, 2000

fenris, Sounds like you are researching thread development. That's cool. In practical application, performance will only be enhanced if symmetrical processors are available to share the work concurrently. Functionality enhancements can be attained using multiple threads even in single processor environments or when the number of concurrent threads is higher than the number of processors. When long processes are going to run in a UI environment threads can be used to perform the work while allowing the user to interact with the user interface thread. Another reason for using threads is to handle multiple requests in a non-linear fashion as in the case of a web server, chat server etc. "But, that's just my opinion... I could be wrong". -pete

fenris · Jun 29, 2000

I thought in the situation of filecopying, that threads may help speed things up since disk I/O is rather slow and most hardrives can easily handle two or three copies without a significant decrease in performance. I have read four or five chapters from various books, numerous articles and such. Reminds me of when I first started to learn java, I was completely lost

I can do the copying with one thread

So I just have to figure out how to get the other thread involved. I really appreciate the comments and suggestions. Thanks, fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

carpeliam · Jun 29, 2000

Wait... so you're not going to take a look at my idea of passing the array to both threads and flagging indexes as they are started? I think that was a great idea.. and it pangs me to see it go to waste ;o) It wouldn't be hard to implement... or have you thought of something else that fits your needs better? Liam Morley <a href=mailto:lmorley@wpi.edu>lmorley@wpi.edu</a> <a href=

http://www.wpi.edu/~lmorley/>

] :: imotic :: website :: [</a> "light the deep, and bring silence to the world. 
light the world, and bring depth to the silence.

fenris · Jun 30, 2000

I will look at your idea, when I better understand how threads work. Right it's as if I am learning java all over again

Every other language that I used had no facility for multiple threads and it will take time to learn how to use them effectively. Thanks for the help though. fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

carpeliam · Jul 2, 2000

fenris, I thought of a better solution. The old one would need one array of String for input files, another array of String for output files, and then maybe a Vector of Integer for keeping track of indexes. Get rid of all that and just have one Hashtable. The key will be a String representing the input file, and the value will be the corresponding output file. If you create an enumeration loop, each thread will remove the key/value it's currently working on. Then, the next thread won't go looking for that one as it has already been removed. (You could also have it be a Hashtable with File types). One of the bigger benefits of this is the fact that you're not adding to memory, you're releasing it. With two arrays and a vector, you're constantly adding to a vector- creating more memory. With this situation of one hashtable, you are constantly removing items, releasing memory. Sounds better to me. have this in your run method: for (Enumeration e = fileContainer.keys(); e.hasMoreElements()

{     String inputFile = e.nextElement().toString();     String outputFile = fileContainer.remove(inputFile).toString();     // perform file copying } I'm not sure if the enumeration is automatically updated as keys are removed from the hashtable. You might need this instead (but I'm not sure): for (Enumeration e = fileContainer.keys(); e.hasMoreElements()

{     String inputFile = e.nextElement().toString();     String outputFile = null;     try {         outputFile = fileContainer.remove(inputFile).toString();     catch (NullPointerException npe) {         continue;     }     // perform file copying } I think that would work. You'd need to create your Hashtable variable outside of everything- either that, or you'd have to pass it in to the constructor for your threads. Also, you can create as many threads as you'd like- it's completely arbitrary. You might also want to have a timer that starts right before the first thread starts and ends right after the last thread dies. This could tell you to some degree the relationship between number of threads in action and the time it takes to complete file copying. Liam Morley <a href=mailto:lmorley@wpi.edu>lmorley@wpi.edu</a> <a href=

http://www.wpi.edu/~lmorley/>

] :: imotic :: website :: [</a> "light the deep, and bring silence to the world. 
light the world, and bring depth to the silence.

fenris · Jul 4, 2000

Liam, thank you for your indepth response. Now I have to learn about hash tables as well as threads ;-) I have started on the first idea that you gave me. When I get it working, and understand how it works, then I will try the second method with the hashtable. Thanks for your help, I really appreciate it. fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

carpeliam · Jul 4, 2000

well it's a fun problem

) I don't get to deal with threads that much... but I've been dealing with hashtables more than ever lately (usually hashtables of hashtables, just crazy), so I think in terms of them. They're pretty useful in certain situations... anyways, best of luck, and if you wouldn't mind sending me the code when you're all done, I'm interested to see how it turns out

) Liam Morley <a href=mailto:lmorley@wpi.edu>lmorley@wpi.edu</a> <a href=

http://www.wpi.edu/~lmorley/>

] :: imotic :: website :: [</a> "light the deep, and bring silence to the world. 
light the world, and bring depth to the silence.

fenris · Jul 5, 2000

I'll send you a copy of the code when I am done Liam. Thanks for the insight. Might I ask what you are using hashtables for? In my wanderings through the java literature I have come accross them but I didn't really get into them. Thanks.... fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

carpeliam · Jul 5, 2000

Well I'm using them for a few different things right now. Here's one use. I'm trying to compare two different sets of data. These are in text files, but the data is sorted differently in the two different documents; and sorting would be a pain... and as I don't want to see two identical documents (I just want to see the errors), that's all I'm returning. As I go through the data, I put it into two sets of hashtables. Then, after I've gone through all the data, I go through one hashtable, removing the data in both and testing if its equal. Seeing as how myHash.remove(data) will remove the data at any point if it exists, the hashtables don't have to be sorted in the same manner. If the second hashtable doesn't have a particular item, it generates a NullPointerException when you go to remove it. That null pointer is an error, and I print it out (along with increment an error counter). After I've gone through the first hashtable, I go through the second one. Seeing as how I've been removing data all this time, I should theoretically have nothing in the second table if there are no errors. If there's anything in the second hashtable, I print it out and report the error. Hashtables are the ideal container component for this because I'm working with name/value pairs. Another use is that I'm instantiating objects from reading a table and returning the final product as a Vector. I read in the data and plug it into a hashtable for easy reading. Each key represents the name of the attribute to set, with a corresponding value. Those are just two uses... hashtables can make life easy

) Liam Morley <a href=mailto:lmorley@wpi.edu>lmorley@wpi.edu</a> <a href=

http://www.wpi.edu/~lmorley/>

] :: imotic :: website :: [</a> "light the deep, and bring silence to the world. 
light the world, and bring depth to the silence.

fenris · Jul 6, 2000

Thanks for the explanation Liam, I appreciate. fenris <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

fenris · Aug 9, 2000

I finally got around to writing the base class of the threaded filecopying utility. I haven't implemented the above suggestions regarding copying an array of files from one section to another. I am more concerned right now with evaluating the performance of the Threaded copy class versus the non-threaded. //================================================= import java.io.IOException; public class ThreadedCopy extends Thread { private String fileToCopy[]; //as an array of two elements, the source string and the destination string "c:\this.txt" "c:\backup\this.txt" public ThreadedCopy(String file[]){fileToCopy = file;}     public void run() { long startTime;   // Starting time of program, in milliseconds.     long endTime;     // Time when computations are done, in milliseconds. double time;      // Time difference, in seconds. startTime = System.currentTimeMillis(); try{ new CopyOneFile(fileToCopy); //Class that copies one file from source to destination } catch(IOException ioe){} try {                 sleep(10); //sleep for 1 second             } catch (InterruptedException e) {} endTime = System.currentTimeMillis();     time = (endTime - startTime) / 1000.0;         System.out.println("the Run time in seconds was:  " + time + " for " + fileToCopy[0]); }      } //============================ I am not sure that is the proper way to time this sort of thing? If this is the proper way, then it works well! I tested it against a class that copies three files (8MB each) one after the other and obtained a time of 17.5s. The threaded version doing the same three files achieved times of less then 12s (the lowest time of five runs). Any advice on how to improve the class would be appreciated... Troy Williams B.Eng. <a href=mailto:fenris@hotmail.com>fenris@hotmail.com</a> <a href= > </a>

Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

threads and file copying

fenris

Programmer

carpeliam

Programmer

fenris

Programmer

carpeliam

Programmer

fenris

Programmer

palbano

Programmer

fenris

Programmer

carpeliam

Programmer

fenris

Programmer

carpeliam

Programmer

fenris

Programmer

carpeliam

Programmer

fenris

Programmer

carpeliam

Programmer

fenris

Programmer

fenris

Programmer

Similar threads

Part and Inventory Search

Sponsor