Deploying iSCSI SAN for VMWare - topology considerations 1

[Stevehewitt]

Hi Guys,

I have a single bus physical topology for our data network all using HP ProCurve 29xx switches. This is a fine little 10Gig backbone network for simple data (and some VoIP with CoS) and a handful of VLAN's - however on the budget given we could not deploy any redundancy.

Our SAN for our VMWare farm has been fibre channel, which whilst expensive has served us very, very well over the past few years. We're looking at replacing our SAN, and seems that we are stuck with iSCSI. I'm fine with this, however it will require us to beef up the Ethernet network to accomodate very high avaliability.

My worries are around deployment of RSTP / MSTP and the recovery time it would take. Whilst <10 seconds is fine for data, and whilst annoying for voice it's not critical (can always redial) I am very wary about a VM being disconnected from the VMDK for this period of time. Would it not create a BSOD on the VM?

If a simple RSTP implementation is not sufficient to prevent VM failure in the event of a cable or network disaster, how else can I implement a more resiliant network? After RSTP I'm only aware of high-end solutions like IRF which is going to be VERY expensive for us to deploy as our current switches don't support IRF!

Anyone else got a redundant iSCSI network for VMWare hosts that can match a dedicated FC mesh that doesn't use IRF or very high end / propriatory protocols?

Cheers - Steve

Steve.

"They have the internet on computers now!" - Homer Simpson

 

[cajuntank]

I did something similar just a week ago to create redundancy on my network. I too use HP switching gear and had some 2910al switches I was using a my tops of the rack switch for servers and SAN connections. What I did was replace the 2910al switches with HP's new 3800 series switches. This series of switch is truly stackable making it one switch, has better throughput compared to even their bigger 5400 series chassis, gives you 10Gb SFP+ ports, and there is no special protocol to configure or setup as again, its logically 1 switch even though you might physically have 2 or 8 in the stack. My SAN nodes have 2x1Gb ports on them which I have set to LACP, then I have each port plugging into a port on both switches with each of those sets of ports in a LACP trunk, so I can maintenance a switch and have no downtime. I also have minimum, 2 NIC ports on each server dedicated to the SAN subnet, so I can have a NIC or cable fault happen there and still be up. I use this new 3800 stack as my core switch and have 10Gb LACP trunk over two connections to a 5400 series switch I use as my access switch for my users, phones, access points, etc...

I looked at doing everything at 10Gb for servers and SAN, but still a little too much money for me right now.

 

[Stevehewitt]

Thanks cajuntank - I'll start looking into the 3800 series. The 29xx's are excellent value for money but the stackability of the 38xx's sound like a huge advantage when looking at poor man's redundancy...!

How have you found resiliance in terms of RSTP? Coming from a FC only background I'm worried about the impact a RSTP correction would take - say 6 seconds - when using iSCSI over ethernet. E.g. Do you know what happens to VM's in the event of a 6 second network outage when using iSCSI? Do they BSOD or does VMWare take this into acount and the VM's just... well, pause?

Was looking into the IRF stuff HP do but I'm wondering if that's overkill and RSTP is fine for iSCSI? Mainly as I'm going to have a hard sell trying to get the kit HP put's IRF on without a solid business case!

Cheers

Steve.

"They have the internet on computers now!" - Homer Simpson

 

[cajuntank]

Again, with the stack, there's no need for any layer 2 redundancy protocols like STP, meshing, etc... and as stated previously, since there are multiple connections made by each SAN node, each Server, and multiple switches, there is something that will be connected always for redundancy purposes let alone additional throughput it gives you.

I use HP's Lefthand SAN equipment and one thing I have per their documentation for Windows is a increased timeout value for iSCSI in the registry of my servers. I run Hyper-V multi-node clusters attached to my SAN and have never encountered anything anything "hinky" with my VMs in that regard. The added timeout value is more for a buffer if you are oversubscribing your VM host server's NIC to the SAN. So for example, if you have a beast of a server box with all the processing power and RAM for 100 VMs, but you are trying to run all those VMs across 1 NIC to your SAN...chances are, it's too much for the one NIC. iSCSI is extremely tried and true, so I would not worry too much about making the shift as long as you follow best practices for your SAN equipment.