Shared Storage for small Citrix XenServer Pools
Haveing Shared Storage in a small type lab environment is what many have around these days. The profile of such a user is that they may over time need team this XenServer pool somewhere (with properly funded shared storage), but they primarily use a XenServer under their desk. They now want to put another XenServer under their desk and share out the VMs.
Example scenario
Consider 2 test engineers who each have a Dell Precision 380 with 4GB memory. Although these machines are getting a bit old, they are capable of running XenServer, and can each run <4 VMs.
A new project comes along which requires greater use of Windows 2008 R2, and Windows 7, but there is only budget to buy 1 Dell T3400 with 8Gb of memory and faster processors. One tester is all smiles, and the other one ends up with 2 Dell 380s and wants to pool them so that he/she can (manually) load balance the VMs.
Bearing in mind that in 6 months time there may be budget to upgrade the second tester, it would be useful if there were some pre-defined shared strorage solutions that might be adopted.
At present I suspect that many people assume that there is not a solution and either a) waste time inventing a “new” storage solution, b) waste time exporting and importing VMs, or c) waste resources by asymmetrically loading their XenServers.
In order to flush out any diverse ideas on how to create shared storage, I recently distributed a survey asking people to share their experiences and to let me know what shared storage solutions they are using. This document collates the feedback received.
Possible solutions
The “correct way”
Use an Equalogic, NetApp or similar appliance.
In my view this is an “enterprise class” solution is outside the scope of this document.
However, you should use something like this if a) you plan to run a production work load, b) any disk, data or VM loss would be expensive, c) you plan to do any performance measurement.
Advantages
– Optimum performance with resilience to disk failure.
– Supported by XenSource, and by hardware manufacturer.
– This is usually the best solution in the long run.
Disadvantage
– Initial cost.
OpenFiler “loopback” VM
The idea here is that you install a pre-built OpenFiler appliance VM on some local/un-shared storage on one of your XenServers. As well as a system disk, the OpenFiler VM has an large additional disk that will be used to provide the shared storage. The OpenFiler appliance then shares its large disk using NFS, and the XenServer uses this share to create an NFS based Storage Repository that can be shared.
This approach is fully described in “XenServer_Demo_and_Evaluation_Setup_Guide_v5.5.1” (CTX118735).
Advantages
– Other than disk space this is almost a no-cost solution.
– It is well documented in CTX118735.
– XenSource make extensive use of OpenFiler servers when testing XenServer.
Disadvantages –
– This solution is probably the least resilient to failures.
– Not fast.
– XenSource do not believe that OpenFiler would handle any significant stress .
– The OpenFiler VM must be running before any other VMs can start, and in some cases it may be necessary to “repair” the shared SR before other VMs can start.
OpenFiler on a re-purposed PC
This is very similar to the OpenFiler “Loopback” VM (above), but in this case OpenFiler is installed on a physical device such as an old PC or server.
Download Openfiler from https://www.openfiler.com/ and install it on an old PC or server. OpenFiler can then be used to host an NFS server that is suitable use with XenServer.
The installation process is described at https://www.openfiler.com/learn/how-to/graphical-installation, is generally quite straightforward, and should not scare those who fear LINUX.
Configuration of OpenFiler for use with XenServers is probably best described in “XenServer_Demo_and_Evaluation_Setup_Guide_v5.5.1” (CTX118735).
The cost of SATA disks often makes it worth installing a SATA disk controller card if new or additional disk space is required.
System 3 are currently testing a Dell 380 to host 300Gb of shared storage for a pool of 2 Dell 380 XenServers. No performance data has been taken but the solution has been seen to work.
Advantages
– It’s easy to set up.
– A repurposed server, or the inclusion of a RAID card could offer better speed and resilience when compared with an OpenFiler “loopback” VM.
– XenSource make extensive use of OpenFiler servers for testing XenServer.
– Less demanding on older hardware
Disadvantages
– XenSource do not believe that OpenFiler would handle any significant stress.
– Old hardware may not be reliable or supportable.
FreeNAS
In many ways FreeNAS is similar to OpenFiler in that it is a cut down version of UNIX that has been configured for maximum NAS (Network Attached Storage) performance, and minimum exposure to UNIX.
One respondent who has tried FreeNAS said “… its setup can be completed in 10-15 minutes …. OS footprint is just 64MB, and boot to OS is just a less than 20sec process. The maintenance is almost zero.“
Advantages
– It’s easy to set up.
– Less demanding on older hardware.
Disadvantages
– Old hardware may not be reliable or supportable.
LINUX on a re-purposed PC
Instead of installing OpenFiler on a re-purposed PC, most people who responded to my request for information used Linux to serve NFS; Ubuntu was the most popular distribution.
Installing Ubuntu is described https://help.ubuntu.com/9.04/installation-guide/i386/index.htmland https://help.ubuntu.com/community/Installation
Installation of NFS on Ubuntu is described here https://help.ubuntu.com/community/SettingUpNFSHowTo#Minimalistic%20NFS%20Set%20Up
The cost of SATA disks often makes it worth installing a SATA disk controller card if new or additional disk space is required.
Linux distributions that have been seen to work.
– Ubuntu with an EXT4 filesystem, serving NFS.
– Susie with a Razor filesystem, serving NFS.
– Fedora with an XFS filesystem, serving NFS.
– In System 3 we use an old Fedora 6 NFS server to provide shared storage to some of our XenServers, and iSCSI for VMware Virtual Infrastructure.
Advantages
– A repurposed server, or the inclusion of a RAID card could offer better speed and resilience when compared with an OpenFiler “loopback” VM.
– Of those who responded to my request for information, this was the most popular solution.
– Depending on your hardware, this maybe the fastest solution for providing shared storage for small XenServer Pools. One respondent has done some performance comparisons using an HP Prolient DL380 (2GB mem) with an HP Smart Array P600 SAS RAID Controller.
– Ubuntu with an EXT4 filesystem – 225MBps.
– Susie with a Razor filesystem – Very good.
– Fedora with an XFS filesystem – Good.
– Free BSD with a UFS filesystem – Average.
Disadvantages
– Old hardware may not be reliable or supportable.
– Some people aren’t comfortable with Linux
NFS service on a Windows 2008 R2 server.
Windows 2008 R2 now includes an NFS server. It should therefore be possible to host an NFS service on an existing Windows 2008 R2 Server.
In addition to the full graphical version of Windows 2008 R2 Server, there is also the lighter weight Core product which may offer increased performance for a given hardware platform.
NFS services have to be installed separately and the following documents describe how to install and configure NFS on Windows 2008 R2.
https://technet.microsoft.com/en-us/library/cc753302%28WS.10%29.aspx
and on Windows 2008 R2 core
https://blogs.technet.com/jorke/archive/2008/09/10/configuring-nfs-on-windows-server-2008-core.aspx
Advantages
– Some people feel more comfortable with a Windows solution
– You may not need any additional hardware as you should be able to add NFS services to an existing Windows 2008 R2 server.
– Windows 2008 R2 Server may also be used to create shared storage for Hyper-V clusters.
Disadvantages
– If using old hardware, it may not be reliable or supportable.
StarWind Free iSCSI Target on a Windows.
Another solution that uses a Windows PC or Server is StarWind iSCSI Target. The free version, which supports up to 2 TB, can be downloaded from https://www.starwindsoftware.com/free
“StarWind Free software is provided only for personal use, demoing, training, or test and development and may not be used for commercial or production purposes”.
Advantages
– Some people feel more comfortable with a Windows solution
– You may not need any additional hardware as you should be able to add StarWind to an existing Windows PC or Server.
– StarWind may also be used to create shared storage for Hyper-V clusters.
– The inclusion of an iSCSI based Storage Repository allows you to enable XenServer’s High Availability option. While you are unlikely to need this in a small XenServer pool, it may be useful if you need to include HA in your testing.
Disadvantages
– If using old hardware, it may not be reliable or supportable.
– Some people feel more comfortable with a non-Windows solution
Windows Storage Server 2008
https://www.microsoft.com/windowsserver2008/en/us/wss08.aspx
One respondent has just started using Windows Storage Server 2008 on a Dell T3500.
The primary purpose of this is to provide clustered storage for a Hyper-V / SCVMM development environment, but it may also be an option for XenServer.
Advantages
– Some people feel more comfortable with a Windows solution
– Developers and Testers may have a suitable MSDN licenses that would allow them use this solution in a development or test environment.
– Windows Storage Server may also be used to create shared storage for Hyper-V clusters.
– The inclusion of an iSCSI based Storage Repository allows you to enable XenServer’s High Availability option. While you are unlikely to need this in a small XenServer pool, it may be useful if you need to include HA in your testing.
Disadvantages
– Some people feel more comfortable with a non-Windows solution
– I have no feedback on performance to date.
Buffalo TeraStation appliance
One respondent has tried a Buffalo Terastation but did not continue as write performance was too weak.