Optimizing Compression

Getting compression to work on Arcserve is frustrating and time consuming. I have some hints. These are based on V7 (Arcserve2000) and V9 BrightStor; and HP DAT40e (20gb native, 40 gb compressed). DAT40e bios C305 (current) and Adaptec 29160 scsi card, current bios 3.10.0

BOTTOM LINE---Use software compression on the local (source) computer-- the computer you are backing up. This is an option that you select from Arcserve Manager, backup screen, source tab. Right click on the disk-- it will say local options. Select "compress files before backup). Make sure compression is turned off on the HPDAT40.

1. Standard streaming tape drives operate at one speed. The HP DAT40 streams data to tape at 3 mb/s (180mb/m) for 110 minutes and thatÆs it. 110 minutes x 180mb/m = 19,800 mb. If you have bottle necks at the source computer or on the network and deliver less than 180mb/m to the tape drive, it doesn't slow down-- it just rewrites the same old data (I think) and wastes space. You'll get less than 20 GB. If you compress what you send to the tape drive (or let it compress) each 180mb/m that it writes is really 200-300mb of disk data.

2. You can watch the mb/minute frame on the job properties window in Arcserve as it runs. If you are running at 180 mb/min you aren't compressing; if you are running 340 mb/m you'll get 37,400 mb on the tape. You don't need to run an entire job to see how you are doing.

For most modern computers (1-2 Gigahertz cpu's) software compression at the source computer is faster than the hardware compression of the dat40 (these are old) and you decrease traffic through the network, a potential bottleneck. You must never run hardware compression with software compression, though-- this will slow down the throughput.

3. To optimize your compression settings and to prove these things to yourself (I'd be interested in having people confirm and correct this) get a copy of HP StorageWorks Library and Tape Tools (free download from HP associated with the DAT40 section of their web site.)
You can test the hardware compression of the HPdat40 using this software, just to prove it works. Another useful test creates large test files of known compressibility. Make one of about 10GB at 2:1 compressibility. Copy it to your source computer.
Use blank tapes for the tests (Arcserve | Device manager| quick erase). Format them as part of the test.

First, use standard hardware compression (this is turned on by default, and you can confirm this by looking at the device details in Arcserve). Run the backup and watch the speed. Anything over 180mb/m is compressing data. Speed x 110 minutes tells you what you would have totaled on a 20/40 gb tape.

Then try the same thing with source software compression. "Quick erase" the tape. Right click under details of the tape drive, and turn hardware compression off. This appears to be semi-permanent. Compression will remain off even after you power the tape drive on and off. And even the HP Software can't turn it back on permanently (it only turns it on for the purpose of the HP tests). Arcserve can turn it back on permanently, though. Just do the right click on a blank tape (or quick-erased tape) and toggle the choice. The details screen in Arcserve shows you the compression setting of the drive. BE CAREFUL, HERE. I am able to turn hardware compression on and off at will using Arcserve. I don't want people to get their HPDAT40 stuck in the non-default, compression-off state.


When I ran these tests on an HP software generated test file (2:1 compressibility) I achieved rates of 305 mb/m using HP DAT40e hardware compression (about 33gb on a tape)
and 350 mb/m using software compression at the source (through Arcserve)-- 39.5 gb on the tape.

I didn't test the other option (software compression in the global options) because I think that in Arcserve V7 and V9 this means software compression is done at the receiving computer (the one with the tape drive) and it makes more sense to compress before going on the network.

Also, you won't achieve these rates on real data that includes a lot of image files (like PowerQuest Drive copy, etc) because these seem uncompressable. But a standard Win2000 server backed up at 350 mb/m using local (source) compression (with image files excluded from the backup) and did put 39.5 gb on the tape.