NB4.5 - SAN - DELL 136T - Exchange 2000 cluster slow backups

[psraj]

Hello Gurus,
We are having a DELL EMC 4500 SAN, DELL 136T tape library with HP tape drives, Netbackup 4.5. The Exchaneg 2000 server is an active active cluster connected to the SAN.
My problem is when the Exchange backups run the backup through put is very slow, like 5MB/Sec.
We have other servers such as the file server cluster whihc is also in the SAN, but the backup through put is better, like 10MB/sec. Is any one using a similar configuration ? Do you get better backup speeds ? The Veritas engineer has said the slow speed is due to Exchaneg being backed up over the physical LAN insted of SAN. He says since Exchaneg is a cluster, the cluster virtual names cannot be backed up using SAN. Any ideas ?

 

[PGPhantom]

Input, Input, I need more input.

I had a similar issue and my PV136T is now up to 15MB/sec. How many drives int he PV136T?
How are they connected, i.e. all on one channel, multiple channels, single server, multiple servers etc?
What controller?
What version of drivers?

 

[psraj]

Hi PGPhantom,
We have one master media server connected to the SAN with dual HBAs. The HBAs in all servers are Qlogix 23xx.

There are 4 HPUltrium LTO drives in the library. The library is dual channel connected to the SAN.

The Netbackup 4.5 itself is patched to teh latest version of pacthes. The drivers running in the Exchange server are Veritas 4.0.1381.1 (Dell PV110T LTO Drive).

The Qlogic drivers are Qlogic 8.1.5.20.

Please let me know what more details you need for this.
Thank you.

 

[adpwl1]

A couple of things:

1) what kind of Exchange backups are you doing? Mailbox level? or just the info store?

2) You say the Exchange is backing up over the LAN. So you have a NBU Client on there. What is the Ethernet speed? You should hard code on the NIC on both the Master Server and Exchange Server to 100mbps(or higher if possible) and FULL Duplex.

3) You say the 136T is SAN connected. So you have a Pathlight in there. The Pathlight Router could be another place to look. We have not had good luck with the Pathlight Routers in the past.

4) What are the specs on the Exchange Server? ie, model, memory...

In my experience, Firmware and drives play a big part on performance. You might want to makesure you are at the latest for everything.

 

[psraj]

Hi adpwl1,

1) We do only Information store level backups. Not mailbox level backups.

2)The Exchange 2000 server is running the Netbackup client, and agents for Exchange 2000. It is also a media server. All NICS are configured for 100Mb Full Duplex. The Exchaneg is also in the SAN with dual Qlogic 23xx HBA. The cluster storage is in the SAN. Can you explain why would the Netbackup use the physical LAN when the server is SAN connnected ?

3) The Exchaneg servers are Dell PE6350 with dual PIII 500Mhx CPU, and have 2GB RAM.

4) As per the Veritas engineer, all drivers are updated to the latest version.

5) We dont use a Pathlight router. We have dual DELL Brocade silkworm DS-16B switches.

 

[adpwl1]

psraj, let see if we answer some of these for you.

1) You say your exchange server is also a media server but not using the SAN attached drives. That is probably because you didn't purchase the Shared Storage Option from Veritas or you have all 4 of your drived deticated to the Master server. I would look at getting SSO so you can share all the tape drives on the SAN with the Master and other Media servers. Your Veritas Engineer should have explained this to you. If not, ask him about SSO.

2) As far as the Pathlight router, you are using them. The are internal to the 136T. It is doing the translation from SCSI to Fibre Channel so you can attach the library and its drives to the SAN. Your `136T is really an ADIC Scalar 100. Check it out on the ADIC web site.

I sell and configure Veritas/ADIC/STK all day and have worked for Dell.

If you want, just ask and we can discuss further.

 

[JanEnEm]

Why do interesting discussions always end like this one?
Could one of you guys shine a light on the end of the story?
Was it SSO or did you find other things?

Jan Meijer
COMPAREX Nederland BV

 

[psraj]

Hello List,
It is my fault I did not update this chain. In fact we have made a much bigger dissaster with our tape library in the past week. All scheduled backups have started failing continiously again. All the 4 HP drives continuously switch to mixed mode. This is not a new problem for us. We have had this problem right from the day one of the 136T installation with Netbackup 4.5. DELL has changed 2 entire 136T librarires for us in the past 4 months. Veritas engineers managed to to get the backups run continuosuly for about 2 weeks last month. The only problem was slow backups with Exchaneg 2000. However, now we are back to square one. Interestingly, the last Veritas engineer who visited us was the best in the lot so far. However, he himself is now reluctunt to further trouble shoot this. He told us "the whole thing is bad" and we need to talk to his management to get another engineer or whatever other decission. In fact the same engineer told us they had similar problem with 136T and NB 4.5 in another clinet site, and eventually the clinet returned the entire product. We have another team of Veritas engineers coming next week. I will let you know the outcome.

 

[PGPhantom]

Our PV136T's are connected via SCSI so that is where the difference is. As I mentioned, I get a sustained rate of over 15,000 KB/sec per drive - This was only after downgrading the Adaptec 39160 Drivers as the latest version has some serious flaws. Up until that time we were only getting around 800-1,500 KB/sec.

From my experience and knowledge ... The pathlight router has been problematic and the solution has not always been - Use the latest version of firmware, BIOS, drivers etc. That has often caused more problems. Try downgrading versions.

Now - If drives are switching to mixed mode it is because you are likely losing connection with the robotics control. This may be network related and it may be Dell related. I assume you have thoroughly tested DNS etc and you are getting complete resolution at all times in all cases? Use the blclntcmd utility to verify.

PV136T - Notorious problems with te picker - We did get ours relaced as it caused numerous problems - After replacing it, no more problems.

 

[bonehead569]

I had issues with the drives always going down. It was a fiber san with a brocade fiber to scsi router. Storagetech and veritas worked on giving us the run around for months before we figured out that the router was retargeting the drives to different lun's. the storagetech field support guy tried a beta firmware on the router and we never had another problem.