Faster Backups

[jciarlette]

I have a file server with 9,282 small files. The total backup size is 389GB. It takes around 22 hrs. I have 3 LTO-2 drives and HP DL380-G4 server with 4GB ram. All are attached via fiber channel. What can I do to try to speed up the backup?

 

[605]

Do i read this properly? - 9300 files only for 400GB? it should run like a dream.

However, if this means 9Mio, then you better use a raw backup (offline backup, free) or SnapImage (online backup, costs an additional license).

 

[jciarlette]

What is a raw backup? How do I perform that? What is 9Mio?

 

[jciarlette]

The file sizes range from 6k to 300k. The harddrives that the data resides on are not fiber attached. I am backing up over the network (100Mb).

 

[605]

A RAW backup is an image backup of a partition or disk. You must use different ways, depending on your NW client operating system. Unfortunately the Legato literature is very poor. I could help, if necessary.

With the file sizes you meant i guess you mean about 9.000.000 (9 Million) files.

 

[jciarlette]

Yes, that was 9,000,000 files. I would appreciate any info on a raw backup.

 

[605]

Here is an example for Solaris:

For the save set specify: /dev/rdsk/c0t0d0s0

Then create a new directive and use rawasm as follows:

<< /dev/rdsk >>
rawasm: c0t0d0s0

Please adopt this for HPUX.

 

[jciarlette]

Is it any different for a Windows environment?

 

[RMinNJ]

If you say your backing it up over the network and one or the
other NIC is 100 Mbits/sec then that is a bottleneck.
A 100 Mbit/s card will do about 10-12 Mbytes/sec max..not
even enough to drive 1 LTO2 drive to maximum speed. Your
rate posted averages to about 5MB/sec but most likely
burst higher at some points during the backup..it could
burst even higher with more network bandwidth.

LTO2 drives do a good job adapting to the rate of the input
even when the input is slow; your choice of drives is good.

 

[jciarlette]

you were exactly right with the avg speed of the backup. Networker was reporting it as well. I will look at the network portion to see if I can connect to a Gbic port.

 

[elgallo75]

Not sure if you have Gige switches in your lan. You can just upgrade to GIGe card on server and notice dramatically performace results.

 

[jciarlette]

I did upgrade the connection to the client to 1GB. Still did not increase the tape writes. I am looking at Legato's snapimage software to see if that helps.

 

[RMinNJ]

Is this one saveset (393GB) ? Perhaps it can be
broken into several savesets based on top level directory?

 

[jciarlette]

That is one saveset. I have already broken it down from a bigger saveset. It would get too hard to manage to break it down any further because of the file structure.

 

[elgallo75]

You can use directives to break it down.. 393GB is as big as a saveset that I would do. Look into Directives as a way of grouping A-L and those sort of things..

 

[jciarlette]

I will check into directives. Not too familiar with the syntax. Will try though.

 

[RMinNJ]

..and your saying that you have Gigabit Ethernet on the
backup server and on the client and you notice little
improvement? You should get above 5MB/sec at some
point.

If you really have 9200 million files, yes, you may need
to back it up differently.

 

[jciarlette]

yes, the client and server both have Ggiabit Ethernet and it is backing up around 5-10MB/s. And there are roughly 9,200,000 files.

 

[RMinNJ]

There was some rumor, and I have not seen it that some have
gotten some improvement on Windows backups with the ignore
directive which tells it to not look for .nsr (or nsr.dir)
files during the backup. That is, ignore local directives.

I tried this on some our backups and saw maybe a 2 to 3%
improvement in speed...maybe. Directive was of the form:

<< "D:\" >>
ignore

If this is a windows backup, you may want to look into
the windows change journal. In theory it should be faster
on incrementals ...reading the change journal instead of
walking the file system. Still the full backup is the same
I believe with it zeroing out the change jorunal.

Snapshots are probably the real answer here though.

 

[emcaixlegato]

You need to make sure that you have updated your client configuration for the addition of the Gb network. In the preferences, "server network interface", you put in your Gb interface server name. For example, in our network, the Legato server over a 100Mb is rs6kpsd5 and over a 1000Mb it is rs6kpsd5s. If you don't do this, then you are still using a 100Mb ethernet for communications between your server and client.