Virtualization Clustering: The IT Pipe Dream

Written by: Steve McMaster

We’ve recently started looking into various types of clustering. We would like to build ourselves a “cloud” that we can run VMware (or some other type of virtualization) on top of. After a couple weeks of investigation, though, there doesn’t really seem to be anything out there that does what we want, and actually works.

We started by looking at some of the actual “cloud” implementations out there. We’re big fans of Ubuntu, so we started by looking at Eucalyptus, which is the cloud implementation included with Ubuntu starting with 9.04. Eucalyptus, however, doesn’t do any sort of load balancing of the virtual machines. It is promising, but right now it’s not quite what we’re looking for.

We also looked at Enomalism. Due to hardware constraints, we don’t have support for “hardware virtualization”. Recent Intel and AMD processors have support for accelerating full virtualization of unmodified guest operating systems. The servers we are using for this do not have support for this extension. Therefore, our virtualization options are somewhat limited. Enomalism has support for KVM/Qemu and Xen. However, the Ubuntu 8.04 support for Xen seems to be broken, KVM requires the hardware virtualization extensions, and Qemu is known to be weak on the performance side. So Enomalism is not really an option for us, either.

We found an interesting project, that also looked fairly promising, called Kerrighed. Kerrighed is a set of patches for the Linux kernel that allow a network of computers to appear as one large multi-processor computer. It also supports migration of processes from one “node” to another. We were initially very excited by this project. However, it seems to have several short comings. Namely, if one member fails, you have to reboot the entire cluster. Also, you can’t add a new member while the cluster is running. Kerrighed shows a lot of promise, and I’d like to try it out as an option again once its gotten a little more mature.

As much as we love open source, we understand that there is a place for commercial software too. So, we took a look at VMware’s Infrastructure 3 stuff. Now I must say, VI3 looked really cool. It has a lot of features, like the load balancing we were looking for (called “VMware DRS”), live migration (“Vmotion”), built-in HA stuff. It was really quite a comprehensive product. However, “enterprise” products like that always come with “enterprise” price tags. For just a couple servers, the price was climbing into the tens of thousands of dollars. This included 24x7 support, though.

What it comes down to is that there is no reasonably priced, effective, stable answer to the clustered virtualization question. We’re probably going to be looking into all of this again in the future. If you’re reading this and you’re a developer of clustering software, I challenge you to make this dream a reality.