ricardo1023
Programmer
I do have about half a Terabyte of data to delete. Is there any Windows tool or software that may allow me delete full folders at super speed?
Thank you
Thank you
Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations IamaSherpa on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
Deleting Large Amounts Of Data
- Thread starter ricardo1023
- Start date
- Status
goombawaho
MIS
Probably quicker deleting to boot from a boot cd (Bart PE or other) and do it from there. That removes the windows operating system overhead from the disk you are trying to delete from.
- Thread starter
- #3
ricardo1023
Programmer
Thank you! I have tech people here but I am not tech myself (in hardware). It is a NAS drive on a server, then I can not reboot. Do you have any other idea?
- Thread starter
- #4
ricardo1023
Programmer
Another Clarification. The data to delete actually is distributed in a R5 of hard drives (It is not a Nas drive)
Thank you
Thank you
Do you have a large number of files/folders? If so, probably not much you can do about speeding up the delete process. OTOH, if there are only a few folders and a large number of folders, it may go faster than you think. I know deleting that much data isn't fun. I've been doing the same lately on a server to make room for other data. The largest amount of data we've removed so far is several hundred megs. Takes several hours (1,000's of folders and approx. 3-4 MILLION files).
I too, would like to know if there is a faster way to delete data of this size.
I too, would like to know if there is a faster way to delete data of this size.
if the data is sitting on an external device, you are dependend on the file system it uses and the OS installed on the device... and possibly the network speed as well, for all the information that is being displayed on the remote PC...
a quicker way, would be to log onto the said server (web gui or console) and tell it to delete the files/folders... though it might only take off a few minutes or so...
and the quickest windows tool to get rid of data is: FORMAT
Ben
"If it works don't fix it! If it doesn't use a sledgehammer..."
a quicker way, would be to log onto the said server (web gui or console) and tell it to delete the files/folders... though it might only take off a few minutes or so...
and the quickest windows tool to get rid of data is: FORMAT
Ben
"If it works don't fix it! If it doesn't use a sledgehammer..."
Johnthephoneguy
Technical User
Would it be quicker to move the data you are planning to delete to a seperate hard drive, then you could easily format the drive with no repercussions.
If deleting large amount of data is something you do on a regular basis I think this might be quicker/easier.
JohnThePhoneGuy
"If I can't fix it, it's not broke!
If deleting large amount of data is something you do on a regular basis I think this might be quicker/easier.
JohnThePhoneGuy
"If I can't fix it, it's not broke!
Really? You think moving 1/2 terabyte of data would be quicker than just going ahead and deleting it. Its going to take the same if not more time to move it. Because you need to wait for it to actually be written. Instead of just deleting entries.
I don't think there much you can do to speed it up. Windows is known for taking a long time deleting stuff. Perhaps trying the deletion through a command prompt may be a little quicker but not really by much.
----------------------------------
Ignorance is not necessarily Bliss, case in point:
Unknown has caused an Unknown Error on Unknown and must be shutdown to prevent damage to Unknown.
I don't think there much you can do to speed it up. Windows is known for taking a long time deleting stuff. Perhaps trying the deletion through a command prompt may be a little quicker but not really by much.
----------------------------------
Ignorance is not necessarily Bliss, case in point:
Unknown has caused an Unknown Error on Unknown and must be shutdown to prevent damage to Unknown.
I don't see the problem...doing things of this magnitude take time, do it overnight. It's not like you're doing it every day, is it?
In the time it took to have this discussion .5TB of data would have been long gone.
Tony
Users helping Users...
In the time it took to have this discussion .5TB of data would have been long gone.
Tony
Users helping Users...
vacunita said:
There's perhaps the answer to the big $64K question. Don't expect Windows to be very fast in deletion if you're just going through Explorer. The obvious reason is the Recycle Bin, in doing all the things that Windows does to handle that. The main thing being that the files are indeed moved instead of merely deleted. The obvious answer in seeking speed is to bypass Explorer and do it via good command-line tools.
Unless you're doing a security overwrite in the process of your delete (indeed copying the data size * 7, usually), a file deletion is a very fast process. Speed will be dictated by the number of files involved more than their file sizes. Thousands of files should be no problem with the proper command-line tools.
----------
Measurement is not management.
Or Explorer either, perhaps. I did a test just now.
30,360 Files; 3621 Folders; 2.33 GB used in both cases.
It took about 10 minutes to copy the data to a folder.
6 seconds to delete the folder (i.e. move it to the recycle bin).
17 seconds to empty the recycle bin of the single folder.
17 seconds to delete the same files with a command-line utility.
Full disclosure: The command-line utility I used was one I wrote which is compiled to Win32 (a DELTREE clone). There isn't anything saying that it couldn't be made faster than it already is.
But the question definitely becomes: "What is going on with this OPs computer that deleting 500GB is a painful ordeal?" If I had to guess, assuming the time can scale for the greater amount of data, it *should* take 3648 seconds or about 61 minutes. It should be very reasonable to delete this data over a lunch hour, and much more so if left to be done overnight.
Is this a repeated process, or is something happening to genuinely choke the system with this much data?
(I shudder to think how long it would take to copy this data to another drive, as has been suggested by others in this thread)
----------
Measurement is not management.
30,360 Files; 3621 Folders; 2.33 GB used in both cases.
It took about 10 minutes to copy the data to a folder.
6 seconds to delete the folder (i.e. move it to the recycle bin).
17 seconds to empty the recycle bin of the single folder.
17 seconds to delete the same files with a command-line utility.
Full disclosure: The command-line utility I used was one I wrote which is compiled to Win32 (a DELTREE clone). There isn't anything saying that it couldn't be made faster than it already is.
But the question definitely becomes: "What is going on with this OPs computer that deleting 500GB is a painful ordeal?" If I had to guess, assuming the time can scale for the greater amount of data, it *should* take 3648 seconds or about 61 minutes. It should be very reasonable to delete this data over a lunch hour, and much more so if left to be done overnight.
Is this a repeated process, or is something happening to genuinely choke the system with this much data?
(I shudder to think how long it would take to copy this data to another drive, as has been suggested by others in this thread)
----------
Measurement is not management.
Glenn,
when I have to delete files in that magnitude, I turn off the recycle bin (Trashcan), that way the data is directly deleted instead of moved first and then deleted...
this is helpful especially, when the data is sitting on an external drive (network mapped etc.), as the data does not have to be transferred between the two systems...
Ben
"If it works don't fix it! If it doesn't use a sledgehammer..."
when I have to delete files in that magnitude, I turn off the recycle bin (Trashcan), that way the data is directly deleted instead of moved first and then deleted...
this is helpful especially, when the data is sitting on an external drive (network mapped etc.), as the data does not have to be transferred between the two systems...
Ben
"If it works don't fix it! If it doesn't use a sledgehammer..."
- Status
Similar threads
- Locked
- Question