Backup Solutions 1

[VVVA]

Our company works with lots of images. We have about 3-5TB of images on our production server at any one time. When we are finished with the images, we back them up to DVD's and delete them off the server.

A while back, our HW dept was re-building the array, and we were supposed to be able to keep working while the array was rebuilding. To make a long story short, the server re-booted itself while the array was rebuilding, and we had MAJOR data corruption. It took months to recover.

I'm trying to figure out some way to get a backup solution implemented, and the cheaper it is the more likely I can talk my manager into it. I've been told it would take two weeks to just back up all of our data, (4TB) and so backing it up on a big NAS box weekly just isn't an option. I have heard that it would be possible to create a mirrored server. Is this the only option, & should I just shut up and be happy with raid 6?

 

[kmcferrin]

I'm trying to figure out some way to get a backup solution implemented, and the cheaper it is the more likely I can talk my manager into it. I've been told it would take two weeks to just back up all of our data, (4TB) and so backing it up on a big NAS box weekly just isn't an option. I have heard that it would be possible to create a mirrored server. Is this the only option, & should I just shut up and be happy with raid 6?

I don't know what sort of solution they were talking about when they told you that it would take two weeks just to do a full backup of all 4TB of your data, but they were either mistaken or lying. My previous employer was a hospital, and we had more data than that backed up just from the Radiology department.

One thing to keep in mind about RAID (of any kind) is that it is NOT a backup system. RAID is for fault tolerance, it is useless for disaster recovery (which could be anything from simple data corruption to your building exploding).

The first question to answer is, how much money would it cost your company if they lost that data and it couldn't be recovered. That will tell you how important it is to get good backups of this data, as well as justification for spending money to ensure good backups. Lucky for you, your company probably already has those figures handy based on this last incident.

If the storage device in question is a proper SAN, they usually have their own replication or backup tools available that should make getting a copy a snap. However, you'll probably have to shell out for a second array (preferably stored offsite) that the data is copied or replicated to.

If you're using a storage array that is direct-attached to a server, then you have another issue entirely, but it can be done. On my old LTO-1 tape drives I could easily get 300-350 MB/min backups from a server that only had 4-6 disks in it. That puts your backup at roughly 9-10 days to complete. However, I suspect that your storage array has more than 6 disks in it and would be able read faster than a 4-disk RAID5 array. Also, there are now LTO-2 and LTO-3 drives that can store more data and write much faster than my old LTO-1 drives. You could get a relatively inexpensive autoloader system for $20-$30k that would have two LTO-3 drives (writing to both simultaneously can cut backup times in half) and slots for 10-30 tapes.

If you didn't want to go that route, you could go with a disk-based system. In theory, you can write the a disk array much faster than tape. However, most people configure their arrays for capacity instead of speed (i.e., RAID5 or 6 instead of RAID 10) and end up hamstringing their efforts. But a 15x750GB disk array would set up in RAID10 with a hot spare would give you a little over 5 TB. You'd probably want to go with a couple of shelves with that config to get a little more space, or you could go with RAID 5/6. Price-wise it would probably be comparable to the tape solution, but it would likely be faster.

What you need to do is determine what fits in with your current backup scheme, and find a solution that not only accommodates your 4TB image collection, but also your other systems. There may be some economies of scale there.

Of course, even with a much smaller setup, you can still manage to back up all of your data. I'm guessing that you have a 4TB archive, but most of it is static and you probably only have a couple gigabytes of changed data on any given day, if that much. Right? If so, you could come up with a rotation that does a full backup (even if it takes a couple of days) once a month, and then do differential or incremental backups (or a combination of both) daily in between fulls. Something like a monthly or bi-monthly full backup, weekly differentials, and daily incrementals will ensure that you get all of the data, and you only have to restore from at most 6 backup sets to get back where you started from.