scooter2179
MIS
Configuring two PowerConnect 5424 Switches for ISCSI Backbone.
We had a vendor come in and setup our VMware environment with a SAN. For the ISCSI backbone, they basically just plugged in the 2 PowerConnect 5424 switches stacked and left everything else default. Recently we have been having major latency issues between the VM hosts and our SAN. VMware support says everything looks fine from their end. They even ran a tool called esxtop with the u option to show me. They said more than likely the SAN is overloaded. So we contacted our SAN Support and they say everything appears to be fine. The arrays don’t appear to be overloaded. This is when I began digging into the switch configuration to find out that no configuration was ever done to optimize ISCSI. I plan on configuring the switches on a weekend but would like to run this past some experts to make sure I am not missing anything on the switch side. We have two Power Connect 5424 switches connected through G24. I plan on setting up two ports on the switches as LAG ports. STP is currently turned on, with no flow control, and ISCSI is disabled as well. Also I have seen were you would definitely want rstp on vs stp. One more thing I should add is that our vm hosts have 4 nics each. 2 nics are plugged into switch 1 and the other 2 into switch 2. Our SAN has 4 nics as well and they are split up the same way. Looking at the counters, there is no type of load balancing going on. Only one nic is being used regardless for the vm hosts and the san as well. I have checked the counters on the switches and havent seen any packet loss.
Someone in another forum recommended creating another vlan and putting all the ports that are being used by hosts and the SAN in a different vlan besides the default vlan 1
Maybe a Vlan 10 on both switches
console(config)# vlan database
console(config-vlan)# vlan 10
console(config-vlan)#name iscsi
console(config-vlan)#ip address 172.16.0.10 255.255.255.0 (Different IP for Switch 2)
console(config-vlan)# exit
console(config)# interface range g1 – 22
console(config)# switchport mode access vlan 10
SETUP LAG on both switches
cosole> enable
console# configure
console (config)# interface range ethernet g23,g24
console(config-if)# channel-group 1 mode on
console(config-if)# exit
console(config)# interface port-channel 1
console(config-if)# flowcontrol on
Do I need to set the LAG ports as trunks?
Turn Flow control on for all ports on both switches
console> enable
console# configure
console (config)# interface range ethernet all
console(config-if)# flowcontrol on
Enable ISCSI on both switches
console (config)# iscsi enable
Enable Jumbo Frame on both switches
console> enable
console# configure
console(config)# port jumbo-frame
Enable RSTP on all ports (Both switches) being used by vmware hosts and the SAN (not on the LAG Ports)?
console(config)# spanning-tree mode rstp
console(config)# interface range ethernet g1-22
console(config-if)# spanning-tree portfast
Should i leave RSTP off on the LAG ports?
Any thoughts on if I am on the right path or are there any changes i should consider before implementing this? Any help would be greatly appreciated. Also please let me know if there is additional information needed.
Thanks,
We had a vendor come in and setup our VMware environment with a SAN. For the ISCSI backbone, they basically just plugged in the 2 PowerConnect 5424 switches stacked and left everything else default. Recently we have been having major latency issues between the VM hosts and our SAN. VMware support says everything looks fine from their end. They even ran a tool called esxtop with the u option to show me. They said more than likely the SAN is overloaded. So we contacted our SAN Support and they say everything appears to be fine. The arrays don’t appear to be overloaded. This is when I began digging into the switch configuration to find out that no configuration was ever done to optimize ISCSI. I plan on configuring the switches on a weekend but would like to run this past some experts to make sure I am not missing anything on the switch side. We have two Power Connect 5424 switches connected through G24. I plan on setting up two ports on the switches as LAG ports. STP is currently turned on, with no flow control, and ISCSI is disabled as well. Also I have seen were you would definitely want rstp on vs stp. One more thing I should add is that our vm hosts have 4 nics each. 2 nics are plugged into switch 1 and the other 2 into switch 2. Our SAN has 4 nics as well and they are split up the same way. Looking at the counters, there is no type of load balancing going on. Only one nic is being used regardless for the vm hosts and the san as well. I have checked the counters on the switches and havent seen any packet loss.
Someone in another forum recommended creating another vlan and putting all the ports that are being used by hosts and the SAN in a different vlan besides the default vlan 1
Maybe a Vlan 10 on both switches
console(config)# vlan database
console(config-vlan)# vlan 10
console(config-vlan)#name iscsi
console(config-vlan)#ip address 172.16.0.10 255.255.255.0 (Different IP for Switch 2)
console(config-vlan)# exit
console(config)# interface range g1 – 22
console(config)# switchport mode access vlan 10
SETUP LAG on both switches
cosole> enable
console# configure
console (config)# interface range ethernet g23,g24
console(config-if)# channel-group 1 mode on
console(config-if)# exit
console(config)# interface port-channel 1
console(config-if)# flowcontrol on
Do I need to set the LAG ports as trunks?
Turn Flow control on for all ports on both switches
console> enable
console# configure
console (config)# interface range ethernet all
console(config-if)# flowcontrol on
Enable ISCSI on both switches
console (config)# iscsi enable
Enable Jumbo Frame on both switches
console> enable
console# configure
console(config)# port jumbo-frame
Enable RSTP on all ports (Both switches) being used by vmware hosts and the SAN (not on the LAG Ports)?
console(config)# spanning-tree mode rstp
console(config)# interface range ethernet g1-22
console(config-if)# spanning-tree portfast
Should i leave RSTP off on the LAG ports?
Any thoughts on if I am on the right path or are there any changes i should consider before implementing this? Any help would be greatly appreciated. Also please let me know if there is additional information needed.
Thanks,