Really slow restore performance.. How to troubleshoot?

[gwichman]

I'm restoring a 70GB Exchange restore and it's been running 24 hours and is only up to about 48GB's restored. When watching the drive, the light blinks for a while then stays silent for a long while.. then blinks again.. The portion where it does nothing seem problematic to me as i'd expect the thing to blink all the time.

I know this is a real general question but i dont see any errors in the reports or windows event log. Btw this is netbackup datacenter 3.41 restoring an E2k server across 100Mbps network to another server (both plugged into the same switch). Network throughput shouldn't be causing this but i'm open to any suggestions.

Is there a performance tuning pdf i can read or where do i start to troubleshoot why things are so slow? What are some of the most common reasons that would cause this?

 

[PGPhantom]

Do you have anti-virus software running? I find that this is the biggest issue when doing restores, people forget to diable it and backup speeds decrease by up to 1000%.

 

[gwichman]

nope

 

[FredTheCat]

A common issue in restore speeds relates to the network connection on both the client and the master or media server.
If for any reason, any component is running at half duplex rather than full duplex, you could see this kind of symptom.
In addition to checking the settings of the NICs and the routers/switches, you may want to try FTPing a file in both directions (both PUT and GET commands) of a file several megs in size. If it is slow in one direction and fast in the other, work with your network admins to determine where your bottleneck is.
If you can FTP in both directions at about the same speed,I would ask if you are trying to restore over top of busy files? You might get better performance if you restore to a separate folder then move/copy the files into place during down time.

 

[gwichman]

Well it doesnt seem to be a network issue. FTPing in either direction is extremely fast as i suspected.. anywhere from 4000KB/sec to 11,000KB/sec

It's an Exchange 2000 restore via the agent..

 

[comtec17]

Are your backups Multiplexed? If so, this could be your issue. Depending on where the data resides on the tape, it may be seeking back and forth.

 

[gwichman]

wouldn't the tape light blink more or less constantly if there were the case? i assume it blinks when reading and when moving to a new position.. i dont understand why it stays quite for long periods

 

[bswip]

If you used multiplexig, the light will not necessarily blink on and off at a constant rate. When mpx'ing, the buffers are written whenever they are full. So if you have a few fast servers and 1 slow server, then the slow server's data could be several blocks apart.

We do a different backup for Exchange. We had MS come in and give us a talk on how they do their Exchange backups and also gave us a white paper. Basically it boils down to using ntbackup locally and then using NBU to backup the ntbackup image. This gives you some flexibility in that the backup image is present on your exchange server and you then have a copy of that image on the media. If you have enough disk storage, you could keep one or more copies locally, and thus your restore process would be very quick. We keep 2 copies and perform differential backups M-F. When we do restores from NBU we have a "hot-spare" the we perform the restore to and then can perform a mailbox recovery.

Why is this faster. Well since the backup image is one big file, your backup will be faster than many small files. Also, you will be avoiding the API. We currently backup around 550 GB of Exchange data nightly in about 4 hours.

 

[85235]

What is the no of files for your restore image?
The dead point of netbackup is to backup or restore a large no. of file like over 0.4 million.
May flash backup feature can solve your problem...!!

Rgds