Linux Virtualization Concepts
External
- https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Getting_Started_Guide/index.html
- https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/index.html
Internal
Generic Virtualization Concepts
Virtualization Solutions from Red Hat
RHEL 7 includes a hypervisor and a number of virtualization tools, which allows running guest operating systems, so it can function as a virtualization platform. However, the solution supports a limited number of guests per host and a limited range of guest types. Red Hat Virtualization is an enterprise virtualization solution based on the KVM technology, offering more features than Red Hat Enterprise Linux. Red Hat OpenStack Platform support OpenStack clouds.
KVM (Kernel-based Virtual Machine)
KVM is a hypervisor that only runs on systems with hardware supporting virtualization extensions. It is built into the standard RHEL 7 kernel, and it can run Linux, Windows, Solaris and BSD guests. KVM is integrated with the Quick Emulator (QEMU). KVM is managed with the libvirt API. The virtual machines are executed as multi-threaded Linux processes controlled by tools built in top of libvirt. KVM supports overcommitting, kernel same-page merging (KSM), disk I/O throttling, automatic NUMA balancing and virtual CPU hot add.
Xen
Xen can do full virtualization on systems that support virtualization extensions, but can also work as hypervisor on the machines that don't.
Quick Emulator (QEMU)
QEMU Guest Agent
The QEMU guest agent runs on the guest operating system and makes it possible for the host machine to issue commands to the guest operating system.
libvirt
libvirt is an Open Source framework providing a hypervisor-independent C-based API that can be used to manage KVM as well as Xen. libvirt's clients are the virtualization management tools. The management tools do not need to be on the same physical machine as the machines on which the hosts are running.
libvirt provides access to management of virtual machines on a host. The API can be used to provision, create, modify, monitor, control, migrate and stop virtual machines. Resources such as CPUs, memory, storage, networking and non-uniform memory access (NUMA) partitions can be listed with libvirt. libvirt delegates management operations to the hypervisor, so only operations supported by the hypervisor can be performed with libvirt.
There is an associated libvirtd service which runs on the virtualization host.
virsh, the primary command-line virtualization management tool, is baed on libvirt.
libvirt Network Support
Also see Networking and KVM Virtualization below.
Network Filtering Rules
virtio
virtio is a package that provides KVM hypervisor-specific code, and exposes paravirtualized devices to the guest operating system. virtio is a layer sitting between the hypervisor and the guest. All virtio devices have two parts: the host device and the guest driver. The paravirtualized device drivers allow the guest operating system to access the corresponding physical device installed on the host system, and they must be installed on the guest operating system. Examples of paravirtualized devices: the paravirtualized network device virtio-net, the paravirtualized block device (virtio-blk), which is a high-performance virtual storage device supported by the hypervisor, the paravirtualized controller device (virtio-scsi), and others (clock, virtio-serial, virtio-balloon, virtio-rng, QXL graphic card)
Also see paravirtualization and paravirtualized devices.
- Virtual Function I/O (VFIO) is a kernel drivert that provides virtual machines with high performance access to physical hardware. It attaches PCI devices on the host system directly to the virtual machine, and it enables the PCI device to appear and behave as it was physically attached to the guest. The driver moves device assignment out of the KVM hypervisor and enforces device isolation at kernel level. This is the default device assignment mechanism in RHEL 7. For more details see RHEL 7 Virtualization Administration Guide - PCI passthrough.
- USB, PCI and SCSI Passthrough. RHEL 7 Virtualization Administration Guide - Guest virtual machine device configuration.
- Single Root I/O Virtualization (SR-IOV).
- N_Port ID Virtualization (NPIV). RHEL 7 Virtualization Administration Guide - Using an NPIV virtual adapter (VHBA) with SCSI devices.
Also see physically shared devices.
Storage and Virtualization
Virtualization Host Storage
Storage Pool
A storage pool is a quantity of storage set aside on the virtualization host for use by guest virtual machines. The storage pool is divided into storage volumes which are assigned to the virtual machines as block devices.
In case the pool's underlying storage is directly attached to the host physical machine, the pool is referred to as local storage pool. Local storage pools include local directories, directly attached disks, physical partitions and logical volume management (LVM) volumes. Local storage pools do not support live migration. It is also possible for the storage not to be physically attached to the virtualization host, but accessible over the network. This is the case for networked (shared) storage pools. Networked storage pools consists of storage devices shared over a network over standard protocols, such as NFS, iSCSI, GFS2, etc.
The storage pool is managed by libvirt on the virtualization host. The size of the storage pool may not exceed the size of the underlying physical storage. Storage pools store virtual machine images or are attached to virtual machines as additional storage. Multiple guests can share the same storage pool.
If not configured otherwise, libvirt uses a directory-based storage pool located in /var/lib/libvirt/images/.
Storage pools can be configured as follows:
Storage Volume
A storage pool is divided into storage volumes, which are abstractions of physical partitions, LVM logical volumes, file-based disk images and other storage types handled by libvirt. Storage volumes are presented to virtual machines as local storage devices, regardless of the underlying hardware. Once defined, the storage volumes' paths can be declared into the storage section of the guest virtual machine XML definition.
A storage volume is uniquely identified by an volume key. The format of the volume key depends upon the storage used. When used with block based storage, such as LVM, the volume key may be a unique string of alphanumeric characters. When used with file based storage, the key may be the full path to the volume storage.
There are three ways to refer a specific volume:
- Using the name of the volume and the storage pool.
- Using the full path to the storage on the host physical machine file system.
- Using an unique volume key.
Note that storage pools and volumes are optional, guest virtual machines may operate without them, but if that is the case, system administrators must ensure availability of the guest virtual machine's storage using alternate tools. Storage pools and volumes provide a way for libvirt to insure that a particular piece of storage will be available for a guest virtual machine.
Virtual Machine Image Formats
The virtual machine image file can only be stored on a host file system, such as xfs, ext4 or NFS.
Disk image formats include:
- raw (also known as QEMU raw format) - which contain the content of the disk with no additional metadata. Raw files can be pre-allocated or sparse. Sparse files allocate host disk space on demand, and are therefore form of thin provisioning. Pre-allocated files are fully provisioned but have higher performance than sparse files. Raw files are desirable when disk I/O performance is critical and transferring the image file over a network is rarely necessary.
- qcow2 - image files offer backing files, snapshot, compression and encryption. They can be used to instantiate virtual machines from template images. qcow2 files are typically more efficient to transfer over the network, because only sectors written by the virtual machine are allocated in the image.
Guest-Side Storage
The storage for virtual machines is abstracted from the physical storage attached to the host. The storage is attached to the virtual machine using paravirtualized or emulated block device drivers, deployed within the virtual machine. Commonly used storage devices:
- virtio-blk is a paravirtualized storage device which provides the best I/O performance, but has fewer features then virtio-scsi.
- virtio-scsi is a paravirtualized storage device for guests using large number of disks or advanced features.
- IDE emulated device. IDE performance is lower than virtio-blk or virtio-scsi but is widely compatible with many different systems. Also see emulated devices.
- CD-ROM device. ATAPI CD-ROM is an emulated device. virtio-scsi CD-ROM is also available.
- USB mass storage devicesAlso see emulated devices.
Networking and KVM Virtualization
A virtual machine guest that connects to an external network uses the software network components of the physical host, which are managed by libvirt's virtual network configuration. The host acts as a virtual network switch. By default, all guests on a single host are connected to the same virtual network, named "default". Guests have default direct IP connectivity to all other guests and to the host. However, libvirt network filtering and guest operating system iptables rules apply. Guests have external outbound access via NAT, subject to the host system's firewall rules. From the point of view of the guest operating system, a virtual network connection is the same as a normal physical network connection.
The guest network interfaces can be set to one of the following modes:
- isolated mode - the network won't allow any traffic beyond the virtualization host.
- routed mode - the network will route traffic between the guest and external hosts without performing any NAT. This enables incoming connections but requires extra routing table entries for the systems on the external network.
- bridged mode - the guests are connected to a bridge device that is also connected directly to a physical ethernet device connected to the local ethernet. This makes the quests directly visible on the physical network, and thus enables incoming connections, but does not require any extra routing table entries.
Security and Virtualization
KVM virtual machines use SELinux and sVirt to enforce security.
sVirt
sVirt is a technology included in RHEL 7 to integrate SELinux and virtualization. It applies Mandatory Access Control (MAC) to improve security when using virtual machines.
Steal Time
"Steal time" is the percentage of time a virtual CPU waits for real CPU while the hypervisor is servicing another virtual processor.
A high value means the physical CPU is overcommitted and more physical CPU should be allocated to the environment - or the VM should be moved on a different physical server. The steal time is reported by vmstat st.
RHEL 7 Virtualization Resource Limits
KVM Guests
A KVM guest has two components: the VM definition, expressed in XML, and the VM image, usually maintained on a storage volume in one of the formats described above.
A VM - and implicitly its definition - can be created on command-line with virt-install or from an XML file with virsh define. The VM definition can then be edited with virsh edit and exported as XML with virsh dumpxml.