Virtual bottleneck

[k3lvin]

I have an issue an need some advice. We have recently just P2V's most of our physical servers. Hyper-V has been working very well and seems stable. The only down side is our backup times and one overnight task is taking longer. Some of our backups speeds on virtual machines are half the speed of what they where when they where physical! There is obviously a bottleneck somewhere. Where is this likely to be? Network or SAN?

Can anyone recomended any software to help me trouble shoot this? We are still backing up some physical machines and these backup speeds are fine still, so the problem has to be on our virtual enviroment. Thanks.

SAN:
MSA2324i
One RAID10 array.
3 volumes, 2 LUNS (1.5TB each) and one Quorum (1GB)
4 NICs on SAN connected to the two host servers.

Hyper-V Hosts:
2 x HP DL380G7 (clustered)
2 x 1Gb NICs (Teamed)

Switch:
2 x HP ProCurve 1810G
4 ports on each switch connected via a trunk.
Flow control enabled.
Jumbo packets enabled.

 

[blister911]

k3lvin,
Can we get more information on your environment?

Networking
1. How is the SAN connected to the switch(es)?
2. Are the ports bound together?
3. Do the servers connect directly to the SAN?

4 NICs on SAN connected to the two host servers.

Disk
1. How many drives do you have in the SAN?
2. What speed and capacity are they?

Backup
1. What software are you using to backup the servers?
2. Where is it running? Physical or VM?
3. Is it a disk or tape backup solution?
4. How is it connected into the VM environment?


Any, all, or none of these could be your bottleneck. Likewise, you could resolve the current issue, only to find out there is another bottleneck.

Check the disk write and read queues for latency. Check the switch ports for errors.

Not knowing how you have things set up, my guess is that your disk is the bottleneck. I'm thinking that you're trying to back up several servers at the same time; all trying to access the same disk for information.

If you are, run a single server backup and compare it to the time it takes when run during the nightly batch backups. To get an accurate comparison, you'd have to run it during the same timeframe as it would normally run.






Light travels faster than sound. That's why some people appear bright until you hear them speak.