File Server - 2003 - Good technique for archiving

[bence8810]

Hi

I hope this isnt in the wrong section. We have a file server running on Windows 2003 Server, with 3x74GB SCSI RAID5. The total size is 130GB, and we are soon aproaching the end of it. As there is no budget this year to upgrade, I have to archive it somehow, burn it on DVD, or just copy it to a different low-profile PC after a full backup, to have it as a read-only copy on the LAN. Its a 35 people office only, so this amount of data (docs, ppts) is way to much.

I am looking for some advice how to archive it, if there are any tools which finds the least-used folders, etc. I dont want to separate files from folders, so I will only archive a folder if it is not used at all completely.

I know this may sound vague, but to be honest, I dont even know where to start.

Ay suggestions are welcome,

Thanks

Ben

 

[SDPT]

this might not be the best way but you can with windows 2k3 add more disks located onto other computers to your server, and this way expand (with risk) you data disk, question is are you willing to take this risk?

To add another disk onto a win 2k3 it is quite easy, just add to your server a net folder that it nothing more than the "shared" disk from the small PC, when you later on add more disks to yoru server it is quite transparent to move the data.

_________________________________
IBM CP, MCP, (Cientist, partime)

http://good-times.webshots.com/photo/2761403060074648262gLrhkV?vhost=good-times

 

[bence8810]

Hi

Thanks for the reply. The reason why I wouldnt like to do that is for the fact that I have RAID5 on the server, so any other hardware would cause a slowdown when associated with it. I was thinking a different machine with large but cheap SATA disks, and through logon script assign this to the users as BACKUP drive, or READONLY drive, etc.

So this can be handled rather simply, but the other thing is how to elaborate what is old data and what is new on the server. So basically I am looking for a technique or a software which can show file-duplications, long-ago not accessed folders, and the kind.

Thanks

Ben

 

[SDPT]

http://noclone.net/adwords.asp

_________________________________
IBM CP, MCP, (Cientist, partime)

http://good-times.webshots.com/photo/2761403060074648262gLrhkV?vhost=good-times

 

[bence8810]

Hi

Thanks, I already am evaluating the software. It does find the duplications quite nicely. What a hell of a job will that be to find all these things, and to select which version to delete.

I need to prepare a report, and tell my users to do it themselves.

If there are more suggestions, I am more than ready for them,

Thanks

Ben

 

[58sniper]

Use Treesize to break things down. It usually helps find the worst offenders.

Pat Richard, MCSE MCSA:Messaging CNA
Microsoft Exchange MVP
Want to know how email works? Read for yourself -

http://www.ietf.org/rfc/rfc2821.txt

 

[bence8810]

Hi

That Treesize is one sweet tool, thanks for bringing it to my attention. The Freeware version doesnt support the networked drives, but locally works like a treat.

I am getting closer to being happy

Ben

 

[LWComputingMVP]

I don't like using treesize because it cannot include files it does not have permission to, so the information cannot be fully trusted. I prefer DIRUSE - it's a command line utility from Microsoft with a good amount of flexibility and you can easily pull in the results to an excel sheet. And diruse will include data it doesn't have rights to. You can find it under Directory Disk Usage (diruse.exe)

http://support.microsoft.com/kb/927229

As for solutions, I think it's a bad idea to move this data to a workstation. Instead, I would go get a RAID controller, a couple of SATA drives, and an external enclosure and setup a new mirror of 160-500GB of additional storage. In fact, that's exactly what I DID do. Parts were $350. My labor was 2 hours, including data migration and creating a script that moved all jpgs off the primary drive (they take a LOT of pictures).

And frankly, if a company of 35 users cannot afford $350 when they fill up their existing drive, I would question the competency of management.

And can you elaborate on this:

The reason why I wouldnt like to do that is for the fact that I have RAID5 on the server, so any other hardware would cause a slowdown when associated with it.

It doesn't make much sense to me - but it wasn't CRYSTAL clear to me what the EXACT suggestion was.

 

[djtech2k]

If you need a cheap, but good disk storage solution, I would maybe look at some of the new NAS devices. SCSI drives for rackmount servers are VERY expensive, but you can get an SATA NAS with RAID for around $300-$1500 and get like 500GB-4TB of space. I think I am going to get one soon. I know there are a few out there, but the one I have looked at is made my Lacie.

There are several options:

http://www.lacie.com/products/range.htm?id=10007

 

[LWComputingMVP]

Unless you're going to spend 10's of thousands on a NAS device OR if you do NOT have a domain and need file sharing, cheap NAS devices are RARELY if ever worth it. External storage and a RAID controller are almost ALWAYS cheaper, more flexible, and potentially better performing.

 

[djtech2k]

In this case he said that he is assuming that his budget will NOT be increasing to manage this solution. So, I was trying to suggest a cheap solution that would be feasible. I agree that internal SCSI or a true SAN is the best way, but both are expensive when you do not have the budget. A $500 product that provides 500GB-2TB of storage in RAID would do that.

As for performance, SATA is great but unfortunately the drive speeds have not reached the 15k like SCSI. This would pose a problem if you were running something like Oracle on the drivespace or there were a ton of users accessing it. He said that there are only 35 users and the space was being used by docs and such, so I think this is feasible.

The long-term solution is to get bigger storage on your server OR clean up all the stuff that you do not need. I would suggest a major cleanup and disk quotas if you do not have ANY money. My NAS device solution was if you could get a little bit of money to get you by and if you could not delete any data whatsoever.

 

[bence8810]

Hi

Thanks for all the suggestions. I was also think about the networked SATA Raid box, which we are currently buying for one of our offices, and it will be used for backup there. NTBackup will push a .bkf file to it every night from all servers.

In the office I currently am, we have a backup server and a tape loader, so I didnt think of getting the box here, but it makes perfect sense. Cheap, small, and protected by RAID. Granted, it isnt as fast, but like you said, we have 35 people all together. The problem here is the amount of docs, presentations, videos (promo), etc.

Our drives are 10K only and are more than okay. So I think a 7K SATA drive wouldnt bring us down too far. And anyways, it will be only for the archived files.

Still, stearing back to the original topic, I am trying to see how to evaluate what file to move and what not. But I guess this is just a tidious process which I have to figth myself across.

I sent an email to all users that by next friday I am expecting to have all unneeded files deleted, and archivable files in a special folder, which then I can move.

I kind of like this Treesize, and the account I am running it with does have access to all files. I guess if we ran a secure financial network, the admin wouldnt have access to all files, but this isnt the case here. The other software mentioned at the beginning of this thread really finds a lot of duplicates. Majorty of duplicates were pictures which are used for presentations, and are copied in each project's directory. On top, we have reports in PPT, which are 30-40 MB in lenght. As the producers of these documents are making changes, they name them accordingly ver1, ver2, etc. Just the other day I saw a ver16. I almost lost my mind. Those should be caught somehow as well.

Well, I will try to fight it next week, and see how far I get,

Thanks for all suggestions, and in case you have more, just let me know,

Ben

 

[djtech2k]

I agree that the quickest, most effective, and cheapest storage is with the NAS. If its no SQL and not a lot of users, its perfect for the price. Also, keep in mind that SATA drives are now made in 10k rpm as well. Thats goes for SATA150, but not in SATAII. Even thought SATAII has double the bus speed, the hard drive speeds are still the bottleneck so an SATA150 with 10k rpm would get you good performance. Even the 7200rpm would not make a diff for your use. And when you have money, expand your server or get a robust storage solution.

 

[LWComputingMVP]

How do you figure?

Quickest? Yes.

Cheapest? NO. Show me a NAS device, I'll show you how you can add the same storage for the same price or less directly to a server.

Most Effective? How do you figure? Most cheap NAS do not offer RAID or any upgrade path.

 

[bence8810]

Well, yes, its quickest. But you are right lwcomputing, that if I slap a RAID card for instance to a Linux box and add 2 500GB HDDs, that will be cheaper, as I dont have to buy the NAS closure itself, and we have old PCs around.

As for opening the only file server we have and adding more handware, I would do it for the simple reason that it doesnt have such a huge power supply, and no more bays I think.

Ben

 

[djtech2k]

You can get a NAS starting at $300. To buy new HP SCSI drives they cost like $700-$1000 per drive for like 146GB drives.

I agree that adding storage to the server is best long-term, but he said he does not have the money.

Non-RAID 500GB $300

http://www.lacie.com/products/product.htm?pid=10844

RAID 1TB $800

http://www.lacie.com/products/product.htm?pid=10876

 

[bence8810]

Yes, we have no money. This box is one solution, maybe best, and other solution is a linux box with some large disks, and file sharing with Samba. No OP system costs, etc. Anyways, I think the NAS must be also linux based on a flash-rom, right?

Ben

 

[LWComputingMVP]

Wow.... I COMPLETELY disagree.

You may be right about the HP SCSI drives... but do you think a cheap NAS is using SCSI drives? No... they are using Parallel ATA - MAYBE Serial ATA. In which case, I can spend $20 on a SATA controller, $40 on an eSATA enclosure and $160 on a 500 GB drive at newegg and you have an external drive with the full performance of SATA for $220. more than 25% less than the Lacie NAS box.

Want RAID?
Addonics 4 bay eSATA enclosure $150
eSATA cable $50
RAID Controller $100
2x500 GB drives $280 setup in a RAID 1
Total Price: $580. More than 25% less than the NAS. And a capacity of 3 TB (2.25 using RAID 5) using 750 GB drives.
And a 5 year warranty on the drives vs. 2 year on the lacie.

Unless you are going to buy a HUGE NAS system with 10+ TB and providing snapshot features (like Volume Shadow Copy), that can be expanded like a SAN, and has 24x7x365 support, NAS USUALLY does not make economical sense.

I had a client with a SNAP drive on an NT4 domain. they upgraded the domain to 2003 - the SNAP drive wouldn't work anymore and the manufacturer wanted $200 to upgrade the OS on it so it could join the domain. For $150, I bought 2x160 GB drives and added them to the server. This provided RAID (which the SNAP drive did not) and more than 5x the space for 25% less than the cost of the upgrade.

To put it another way, NAS is a great concept... problem is, it's overpriced.

 

[bence8810]

NAS is only maintenance free which should be mentioned. We will get a DLink one if I remember well for our other office, and it has an embedded linux OS which you can program over HTTP. It has great scheduled timers, you can even create the backups on the local machines, and have ths box download it to its store at a scheduled time. Once you configure it, you dont need to touch it ever again. It also sends emails when there is a problem, you can add users with different privileges, and on top, its also an FTP server.

I think nothing beats that, when you need it at a place where you have no support personnel etc.

When the HDD gets fried, you get an email, the receptionists opens the bay, slides another HDD in, you log in remotely and rebuild.

Well, anyways, this is taking the original topic way off line.

Ben

 

[LWComputingMVP]

There are reasons to get NAS, but in my opinion, MANY more reasons not to - when all you want is the cheap stuff. RAID controllers notify you via email of disk failures as well. Hot swap is nice... but you can get the same functionality for less in an external bay attached to a server... Having a built in FTP server doesn't impress me... Windows has plenty and they are also a security risk (data is transmitted in plain text). And once you configure an extra disk in a server, there's really little IF ANYTHING you ever need to do it... I don't see NAS as maintenance free (Firmware updates?)