The hardening of virtualized environments is growing in popularity. Virtualization has the advantages of isolating services on various slim guests running on a larger server, while hardening provides for enhanced security for both the guests and host. In practice, however, getting the two to work together is not always an easy task as the technologies employed by one often interfer with the other. This is complicated by the fact that there many implementations of virtualization and many degrees of hardening. This guide aims to provide some clarity to the issues and outline some best practices.

This guide looks at virtualization using kvm, xen and vmware under hardening by GRSEC/PaX. For each type of virtualization, we discuss what hardening features work for the host and guests without either degrading performance horribly or breaking completely. This is not a howto on setting up virtualization since that is covered elsewhere; rather, we limit our discussion to just what hardening features ought to be enabled or disable when configuring the kernel of the host or guest operating systems.

KVM (Kernel-base Virtual Machine) provides virtualization on x86 and x86_64 hosts that have the required hardware support (Intel-VT or AMD-V). The host uses a general kernel module (kvm.ko), a processor specific module (kvm-intel.ko or kvm-amd.ko), and a userland utility (qemu-kvm), to run the guests. The guests can be configured to use emulated hardware (full virtualization) or virtio (para virtualization). Paravirt has the advantage of increasing performance and providing a common I/O interface between host and guest. Resources for setting up kvm on gentoo can be found at the end of this guide.

As of this writing, there are no known restrictions on hardening for the guest on amd64 hosts. Test of both x86 and x86_64 guests using either emulated hardware or virtio, with all hardening features, including CONFIG_PAX_KERNEXEC and CONFIG_PAX_MEMORY_UDEREF, have been successfull on amd64 guests. For Intel hosts there have been reports going both ways on whether or not CONFIG_PAX_MEMORY_UDEREF being enabled in the guests causes the guest to run slowly. Currently it is recomended to not enable CONFIG_PAX_MEMORY_UDEREF on Intel guests.

For the host, however, one must disable both CONFIG_PAX_KERNEXEC and CONFIG_PAX_MEMORY_UDEREF. Either of these will set an invisible kernel option, CONFIG_PAX_PER_CPU_PGD, which is know to break kvm. What is actually happening is that the guest's performance is degraded to the point where it is unusable, but doesn't crash, and the host is left with qemu-kvm in uninterruptible sleep (state D when doing ps aux). Only rebooting the host clears the issue.

These tests were done using the 2.6.32 and 2.6.34 branches of the kernel with GRSEC/PaX patch version 2.1.14 and 2.2.0 (see Gentoo bug #328623). However, it unlikely that this problem will be solved anytime soon, which is unfortunate because both KERNEXEC and UDEREF are excellent hardening features.

Xen is an older virtualization technology than kvm, but similar in many regards. It employs a hypervisor which boots a specialize host's kernel (dom0). Once the host is up, it in turn runs guests (domU) ... TODO

VMWare Workstation needs to link precompiled binaries against system libraries in order to function. Because Gentoo Hardened uses more secure functions of GCC, VMWare Workstation cannot link against it. Because VMWare Workstation cannot link, it does not function. In fact, using VMWare Workstation at all on Hardened Gentoo led to a hard system reset.

KVM related resources:

• Setting up KVM on Gentoo Linux
• Using Virtio Drivers in Linux