The future of VSAN – My take

There have been a few posts speculating on the future of VSAN and I for one am looking forward to it with great anticipation. However, I don’t think VMware really know what a hugely transformative technology VSAN could be.

I was lucky enough to attend VMworld 2015 and luckier to be invited to the VSAN pioneer summit, which gave us a real in-depth look at the future of VSAN. I liked what I was seeing but about an hour towards the end of the allotted time I put my hand up and asked why there were no NAS features planned for the future release. I mean it makes sense doesn’t it? Where’s NFS, where’s SMB? I know a linux architect who would love to see this come in.

If you really want to do the software defined storage thing then really go for it. NSX is the current favourite child. its being pushed everywhere, including presence into “competitors” such as AWS. So where’s the love for VSAN? Push this technology and it will really change the datacenter.

First Thoughts.

What if VMware made a VSAN only cluster, no VM’s allowed only storage exports. This would put them in direct competition with Storage vendors and would greatly reduce the cost for storage in the datacenter and allow for a huge amount of flexibility for businesses of all sizes. lets explore this idea more!


Folders (native on the file system) or VMDK’s

VMDK wins. I would think that using VMDK’s instead of folders would be a much better idea. There would be no real changes needed to the VMFS file system to accommodate a much more granular permission structure that would be required by SMB. ESXi could mount the VMDK and write any file system in there. VMDK’s can be accessed by multiple ESXi hosts.

NFS3 – NFS4 – SMB2.x – SMB3

We already know that NFS4 and SMB3 can take advantage of multiple IP addresses (hosts) to provide multi-channel and VMware clusters are, quite frankly, an incredible implementation of clustering technology. Mounting the VMDK to multiple ESXi hosts would allow the data to be taken advantage of  by NFS4 and SMB3 compliant hosts.

SMB2.x and NFS3 prefer to access data through a single IP address or hostname. Now this is easy to implement immediately but if you want to add a bit more intelligence around it, some kind of construct that has a virtual IP that could move between hosts or something like the virtual IP address technology from Log insight clusters. Easier said than done I know but still should be considered.

Redundancy and performance

Kinda obvious, i know, but redundancy would be taken care of by VMware clustering technology. three or four hosts and that’s that taken care of.

Performance on the other hand could be very interesting topic, a complex topic, but still interesting. I would guess in the thousands of IOPs. There would be many factors to consider. Network speed, controller card, SSD speed, SSD size, and so on and so forth. In a future post I’ll look at this again.


As this is only intended to be a storage service the licensing should be one ESXi-VSAN license (I’ve guessed it to be £1,500 but could be as high as £2,000, which I’ve also given as a cost per TB below)

Total Cost

So this is interesting and I’ve decided to look at a couple of real world examples below.

Dedicated Storage Appliance

I have a quote from a major vendor for £198,409.45. This figure gives us 48TB of HDD storage in 64 SAS disks and 9TB of SSD storage in 8 SSD disks (these figures are usable). For this project we decided to use the SSD as a caching layer.  As you would expect from an enterprise storage system it has a good deal of redundancy built-in with 4 nodes to manage the storage and 8 x 10GB Ethernet ports. All in, not bad for the price point and a good system all round.

Dedicated VSAN Cluster

Putting together our VSAN only node, to compete on numbers, I would size it like this: Looking at an HP DL380 Gen9 with one CPU (E5-2623) 32GB of Ram. Two disk pools with 1 x 800GB SSD and 7 X 1.2TB SAS disks each, giving us 1.6TB of SSD cache and 7.5TB of SAS storage (again these figures are usable based of a default VSAN storage policy of 2n). Two 10GB Ethernet ports.

To get the equivalent amount of usable storage as the popular storage vendors array we’d need 7 VSAN nodes.

So for the costs:

Items Storage Vendor VSAN
Nodes 4 7
10GB Network 8 14
SSD Cache Size 9.2TB 10.5TB
Usable SAS 48.5TB 52.9TB
Cost per system £198,409.45 £109,320.40
Cost per TB £4,090.92 £2,066.55

Note 1: I have estimated the cost of the VSAN license at £1,500. If the license were £2,000 then the cost per TB for VSAN would be £2,132.71.

Note 2 : (To be fair) The Storage vendor has extra goodness built-in to accelerate workloads and the hardware will be optimised and custom designed to do nothing but server data.

The above figures, which speak for themselves, are all based on real quotes and would be for an enterprise deployment.

If VMware really wanted this to be everywhere they could address smaller shops by allowing a single node VSAN. Why not; that would allow anybody to get a foot in and expand as their business grows.

So VMware, when will this be a reality for us?

Please let me know what you think and it there are any glaring errors. I’m also happy to discuss any of the above.

Nested Home Lab – Part 12 – Distributed Virtual Switch

As we discussed in Part 2, for our basic lab, we wanted to separate out the networking  in to three parts:

Use Example IP Range
VM and Management
VSAN (vlan 30)
vMotion (vlan 40)

We have already set-up our physical host’s network in preparation to create our distributed switch.

So what we are looking to do here is create our distributed switch, create the port groups (tag them), and add our VMkernel interfaces. Easy Right?

Creating a Distributed Switch

  1. Log in using an account that has permission to configure the environment.Lic-1
  2. Select Home and Hosts and Clusters.AH-1
  3. Click on networking icon, right click on your Datacenter, Select Distributed Switch, Click New Distributed Switch.
  4. Enter in a name for your new switch. Click Next.DS-3
  5. Select Distributed Switch: 6.0.0.  Click Next.
  6. Set the number of uplinks to 2 and un-tick create a default port group. Select Next.DS-5
  7. Check that our setting are correct and select FinishDS-6

And thats the switch created.

Adding your port groups

We’re going to be creating two port groups, vMotion with a VLAN tag of 40, and VSAN with a VLAN tag of 30.

  1. Right click on the Distributed Switch. Select Distributed Port Group. Select New Distributed Group…
  2. Give the port group a name. In this case vMotion. Select Next.DS-8
  3. Set VLAN type to VLAN and the VLAN ID to 40. Leave all the other options as defaults.
  4. Double check the settings you specified are correct and click Next.DS-10
  5. Go back and create a Virtual SAN VLAN, but with a VLAN tag of 30.DS-11
  6. You should now have a vSwitch that looks like this:DS-12

For a basic lab, as we are creating, the networking is quite simple. However virtual networking has come a very long way and with the introduction of NSX it is now effecting the whole data centre. I would recommend you read Networking for VMware Administrators by Chris Wahl and Steven Pantol. While the book doesn’t cover NSX (understandable as its a whole discipline in itself) it is very good and I would highly recommend it.