What SAN to buy for VMware?

[Eddiefdz]

Hello,

Any opinions as to the industry standard SAN solution for running VMware Server or ESX? I am looking to purchase a SAN for VMware but I am not sure which one to look at, please let me know what you think or what you are personally using for your setup.

Thanks,
Eddie

Eddie Fernandez
CCNA, Network+, A+, MCP

 

[ArizonaGeek]

Well fiber channel is the way to go, it seems iSCSI still has some issues until the new version is out. We just bought an EMC SAN from Dell, 4tb of drive space with 4gb/sec fiber. Have yet to rack it up but that'll probably be Monday.

Since EMC and VMWare are now one, EMC would probably the logical choice when it comes to a SAN but that is just my opinion.

I can tell ya to stay away from Compellent SAN. We have a 6tb iSCSI SAN from them and have quite a few problems with it.

Cheers
Rob

The answer is always "PEBKAC!

 

[unclerico]

We are running an EqualLogic iSCSI SAN with our VMWare intrastructure and I must tell you it is absolutely phenominal. The ease of installation, management, and maintenance is amazing (no I don't work for EqualLogic/Dell) with these devices. EqualLogic is in bed pretty tight with VMWare so it would be worth your while to take a look.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[skialta]

Just make sure it's on the HCL for VI3...and be sure that you have dual-pathing enabled.

 

[mrdenny]

My preference would be EMC. We are in the processing of purchasing an EMC SAN (~40 TB in case anyone is interested). Part of what it will be hosting is VMWare. Do keep in mind that a SAN isn't a black box that you can just drop stuff on and it'll run great (sales people do sometimes say that). It does require proper configuration and regular maintenance to keep it running at peak performance.

Denny
MCSA (2003) / MCDBA (SQL 2000)
MCTS (SQL 2005 / Microsoft Windows SharePoint Services 3.0: Configuration / Microsoft Office SharePoint Server 2007: Configuration)
MCITP Database Administrator (SQL 2005) / Database Developer (SQL 2005)

My Blog

 

[koopie]

We have had a SAN (EMC) installed today. As we are new to SAN's and all that gows with them I was wondering what "regular maintenance" tasks are required to keep one "running at peak performance"?
We are going to run VMware ver 3.x on 3 x Dell 2950 servers in conjuction with the san. Any thoughts or opinions would be welcome.

ChrisK

 

[mrdenny]

A few things to keep an eye on:

Make sure that the Peak IO of the LUNs being used and the RAID GROUPs being used are never above the limits of the RAID Groups. You'll want to keep an eye on the queues, MB per minute or second, etc.

You'll also want to make sure that your disks are setup with the block alignment offset setup correctly on both the physical and virtual disks.

I've also seen where it's recommended (sorry I don't have URLs) to put the virtual machiens page files on a seperate high end LUN from the OSs. This way when the OSs need to read it isn't compeating for the disks from the page file writes.

Also everything should be on fibre channel, SATA doesn't have the throughput to host Virtual Machines. Make sure that nothing is put on the disks that host the Vault OS. We had to put stuff on the vault and the system performance suffered greatly for it until we got the volumes off the Vault RAID Group.

Denny
MCSA (2003) / MCDBA (SQL 2000)
MCTS (SQL 2005 / Microsoft Windows SharePoint Services 3.0: Configuration / Microsoft Office SharePoint Server 2007: Configuration)
MCITP Database Administrator (SQL 2005) / Database Developer (SQL 2005)

My Blog

 

[NickFerrar]

We have Dell|EMC CX3-20c SANs, not actually started hosting VMs on it yet but will do early in the new year. As others have said VMWare is owned by EMC now so if you want to err on the side of caution I'd go the EMC SAN route.
We currently don't do any routine maintenance on the SANs, I need to set up the NaviAnalyzer logging properly soon though.
Dell do a managed health check service to (not sure if EMC offer a similar service for pure EMC SANs), whereby an engineer does a site visit and checks the SAN and SAN-attached hosts for problems and verifies the config.

Not sure how other SANs compare in price/performance to the EMC Clariion stuff but unless they're significantly cheaper I'd take the easy option and go with EMC...

 

[johnny99]

We mostly use EMC DMX-4's and HDS USP-V's.
And directors are mostly McData 6140 (now Brocade M6140)

It works fine with VMware.

On the cost side I have no clue, but I gues that we have a fairly low cost per port and per GB.

/johnny

 

[sbruce66]

Hi,
If you are looking for a great SAN/NAS then StoreVault by NetApp. Specifically the S500. We bought a 6TB for 15,000 and it is certified for ESX. We bought it from a company called PBG.

http://www.bennettgrp.com.

By the way this is a far better device then a very expensive EMC and can run NAS and SAN at the same time. Hope this helps.

Geoff

 

[mrdenny]

the way this is a far better device then a very expensive EMC and can run NAS and SAN at the same time.

Wow, that's a statement. Would you like to back that up? How exactly did you come to the conclusion that a NAS with a SAN extender built into it performs better an a SAN device?

Denny
MCSA (2003) / MCDBA (SQL 2000)
MCTS (SQL 2005 / Microsoft Windows SharePoint Services 3.0: Configuration / Microsoft Office SharePoint Server 2007: Configuration)
MCITP Database Administrator (SQL 2005) / Database Developer (SQL 2005)

My Blog

 

[sbruce66]

Denny,

I am wondering how you can make some of your claims. We heard about this product from another company who had 2 EMC Clarions at about 300,000 thousand a piece. They bought 9 StoreVault S500 at 6TB each with Fiber Channel and some with iSCSI HBA. (For about 100,000) They were running 9 Solaris Ultra Enterprises Servers running Solaris 8. They also had 4 Dell servers running linux with Vmware using NFS instead of iSCSI to run esx 3.x. Vmware confirmed we could do this. Also your saying SATA can not handle the throught put of Virtual machines. 2 of the linus servers were running SATA. Each server had 16VM running. Where did you come up with that statement. By the way the company was doing all of this work for the Army as a cost saving measure. Also it looks to me that you are a Microsoft shop, what is your company doing that you need that type of Bandwidth and if you do why are you not using Unix or Linux.

I recomend you all check this product out, call storevault and see how many large companies are using this box as an alternative to Dell or EMC. Unless you are a bank or trading company and need a hugh amount of storage why would you need suck power at a great expense!

Geoff

 

[mrdenny]

A couple of companies ago we had a couple of EMC shelves with SATA drives in them and we got very poor performance on those LUNs when ever we were trying to do anything that was more than one or two threads.

If your VMs boot then don't do much with the disk, then SATA may be a good option to host those VMs. We had some file shares on SATA 4+1 arrays and the performance tanked when ever more than about 3 people were trying to use the LUN. (We only had a single LUN on the raid group for testing.) The queue times on the RAID Group went up and the number of operations being handled per second dropped very low as soon as the second and third person started using the LUN. (I don't have any numbers as I don't work there any more.)

We are a Windows shop (you can probably guess that I work in the Windows world via my sig file). We have a very high load database which needs a ton of IO to satisfy requests, as well as file servers which accept and serve up millions of file requests per day (depending on the hour, hundreds of thousands per hour).

Can you be more specific as to which statment of mine you wanted more info about? I've got a couple of posts on this thread so I tried to address all of it. If you'd like me to dig a little deeper into one of my statements, please let me know which one, and I'll do my best.


Denny
MCSA (2003) / MCDBA (SQL 2000)
MCTS (SQL 2005 / Microsoft Windows SharePoint Services 3.0: Configuration / Microsoft Office SharePoint Server 2007: Configuration)
MCITP Database Administrator (SQL 2005) / Database Developer (SQL 2005)

My Blog

 

[johnny99]

I would have to back Denny up on this.

I am sure noone in our company would think the S500 StoreVault from NetApp is a better SAN disk box then a HDS USP VM or a EMC DMX-4.

But I don't say that a HDS USP VM disksystem is the best in all situations.

A HDS USP VM cost a lot (no clue about the price and I also don't care) but we use them because of a low TCO.

But this isn't a solution of choise for an installation that need 50 TB of disk capacity in most situations.

/johnny

 

[ljcps]

Hi -

The company I work for deals in 40 manufactures of storage. Here are some ideas. Hope it helps.

Check out the following:

- Fujitsu
- NEC / as a sub for EMC (way less expensive)

- Stonefly - if you decide to go iSCSI
- HIFN - Great value if your need is right

You can get some real values from full service product through Stonefly.

Let me know if you need more info on any of these...I have tons.

- lj

 

[sbruce66]

Derry,
Your low cost EMC running 2003 Storage server has a hugh aamount of overhead because of the windows OS. Also the design of smaller EMC boxes. Not a result of SATA. Our S500 runs Ontap a 75 meg OS runninmg in memory. The box was able to handle 20,000 connections using smartbits traffic simulation technology.

We have ArizonaGeek in here who bought a 4TB EMC solution for some dell servers. A S500 definetly would have been a better choice. Now when you get 50TB or above S500 may or may not be the right choice. Depends on what feature set you want. If you want clustering then S500 isn't your box. I will tell you it can handle some serious workload. I have our S500 running VMware Solaris Guest which is running Netscreen Security Manager (NSM) controlling 12,000 VPNs around the US for the Government. It has been running fine without a problem. In addition I am running WhatUp's Gold in another VM to monitor all these bases as well as 4 other VMs.

This box would be great for just running VM, and that is what alot of people from VMware have been telling us. Obviously EMC has it's place and so does the StoreVault but NetApp definetly has some cool technologies. I do however like hitachi and equalogic as they are great alternatives. Also I will tell you the Army is able to backup 54 TB in less then 5 minutes using snapshot technology something EMC really hasn't addressed. EMC is still limited to 16 Snapshots per lun.

 

[NickFerrar]

No regrets here about buying EMC Clariions, the CX3 series are pretty cheap for you get IMO. I work for an IT outsourcing/hosting company so we have a variety of kit attached to it (Windows NT cluster, Windows 2003, RedHat Linux, Solaris etc etc). We currently don't have SATA disks but will do shortly for disk-to-disk backups.

All the attached hosts have local OS drives so I really can't see how there's much of a difference in SAN overhead between Windows and Linux in that scenario.

As for VMWare - we're about to start a big consolidation project ourselves and are still debating whether or not to go iSCSI or FC, regardless I'm pretty sure it will be 15k FC disks for the LUNs though.