XenServer 5.6 Multi-GPU Passthrough for XenDesktop HDX 3D Pro Graphics
This article describes how to configure XenServer 5.6 with XenDesktop 4 HDX 3D Pro Graphics so that a host workstation with multiple graphics cards can be shared by an equal number of concurrent high-end professional graphics users (one user per GPU), thereby reducing the cost per user. Multi-GPU Passthrough is a Tech Preview feature of XenServer 5.6, not intended for production deployment at this time.
HDX 3D Pro Graphics, a feature of XenDesktop Enterprise and Platinum editions, enables the delivery of high-end 3D graphics applications, including OpenGL and DirectX based applications, such as CAD/CAM and medical imaging. Typically these applications are very computation-intensive and require hardware acceleration provided by a graphics processing unit (GPU). To support remoting of high-end 3D graphics based applications, Pro Graphics leverages a graphics card on the host workstation (usually located in the data center) for 3D rendering and optionally for deep compression of the data stream. Because multi-user hypervisors prior to XenServer 5.6 did not support GPU access, HDX 3D Pro Graphics was previously recommended with physical machines only, whether physical desktop machines or blade or rack workstations.
With the Multi-GPU Passthrough feature of XenServer 5.6 it is now possible to map a physical GPU to a virtual machine; in fact, you can map multiple GPUs to an equal number of virtual machines, one to one. XenServer 5.6 with GPU mapping in the direct assign mode is the ideal solution for reducing hardware costs for high-end 3D graphics. Consider a server that has multiple GPUs or possibly (not yet tested) one that is connected to a multi-GPU appliance (such as NVIDIA Quadro Plex S4). In this case, each GPU can be mapped to one virtual machine, which can get access to the GPU and is powered by HDX 3D Pro Graphics. On a powerful multi-core server (such as a dual quad-core Nehalem processor server that would normally support 80 or more regular XenDesktop users running typical office applications), additional virtual machines could potentially run regular XenDesktop for normal use cases not requiring graphics hardware acceleration, subject to performance, thereby further reducing the cost per user.
The Server CPU should be IOMMU enabled for AMD processors or VT-d enabled for Intel processors.
This solution requires nVidia GPUs supporting SLI Multi-OS, such as the FX3800, FX4800 or FX5800. Install the latest graphics driver from NVIDIA. AMD/ATI graphics cards are not supported by the Multi-GPU Passthrough feature.
HDX 3D Pro Graphics utilizes one virtual CPU completely for extracting desktop surfaces, typically 12 to 30 times per second. If Aero is enabled on Windows this requires a second virtual CPU. In addition, at least one vCPU is required for the graphics applications. Graphics applications are often CPU intensive, even with hardware acceleration of graphics rendering provided by the GPU. Hence it is important to configure the virtual Machine running HDX 3D Pro Graphics with at least two virtual CPUs if running on Windows XP or using the Basic mode of Windows, and at least three virtual CPUs if running on Windows 7 with Aero enabled. Instructions for increasing the number of virtual CPUs on XenServer are provided below.