KVM Virtual Networking Concepts

External

• RHEL7 Virtualization Administration - Virtual Networking
• https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/chap-Guest_virtual_machine_device_configuration.html
• https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/sect-Managing_guest_virtual_machines_with_virsh-Interface_Commands.html
• https://wiki.libvirt.org/page/Libvirtd_and_dnsmasq
• https://wiki.libvirt.org/page/Networking
• https://wiki.libvirt.org/page/VirtualNetworking
• https://libvirt.org/formatdomain.html#elementsNICS

Internal

• Linux Virtualization Concepts
• libvirt Network Operations

Overview

This article discusses concepts related to virtual networking with libvirt. The main libvirt server-side virtualization management system component is the libvirtd daemon. Configuring and manipulating networking is one of its responsibilities. Guests can be connected to the network via a virtual network, or directly to a virtualization host network device. The virtual network setup is appropriate for the case where there is significant network communication among cooperating guests. If a guest is primarily intended to serve external traffic, a setup where the guest is connected directly to a publicly exposed virtualization host network interface is more appropriate. A combination of these options is also possible.

Virtual Network

libvirt implements virtual networking using a virtual network switch, which is logically equivalent with a virtual network. A virtual network switch is a software component that runs on the virtualization host, which guests virtual machines "plug in" to, and direct their traffic through. The traffic between guests attached to a specific virtual switch stays within the confines of the associated virtual network. From a guest's operating system point of view, a virtual network connection is the same as a normal physical network connection.

The virtual network can be restricted to a specific physical network interface. This may be useful on virtualization hosts that have several physical interfaces, and it only matters for NAT and routed modes, as described below. This behavior can be defined when the virtual network is created, by specifying "dev=<interface>".

On the virtualization host server, the virtual network switch shows up as a network interface, conventionally named virbr0.

# ip addr
...
7: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 52:54:00:15:ef:87 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
...

In the default configuration, which implies NAT mode, this network interface explicitly does NOT have any physical interfaces added, since it uses NAT + forwarding to connect to outside world. This can be proven by running brctl show on the virtualization host. The routed address is by default 192.168.122.1 and the routing to a physical network interface is done in the virtual host networking layer. libvirt will add iptables rules to allow traffic to/from the virbr0 interface. It will also attempt to enable IP forwarding, required for routing.

The virtual network configuration can be displayed with virsh net-dumpxml:

<network connections='6'>
  <name>default</name>
  <uuid>49a07631-ea20-4741-89ea-b0faa7b42d19</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:15:ef:87'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.100' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

The virtual switch is started and managed by the libvirtd daemon, via dnsmasq processes. A new instance of dnsmasq is started for each virtual switch and the virtual network associated with it, only accessible to guests in that specific network. The dnsmasq acts as DNS and DHCP servers for the virtual network. The configuration file for such a dnsmasq instance is /var/lib/libvirt/dnsmasq/<virtual-network-name>.conf. By default, there is just one "default" virtual network, and its dnsmasq configuration file is /var/lib/libvirt/dnsmasq/default.conf:

strict-order
pid-file=/var/run/libvirt/network/default.pid
except-interface=lo
bind-dynamic
interface=virbr0
dhcp-range=192.168.122.100,192.168.122.254
dhcp-no-override
dhcp-authoritative
dhcp-lease-max=155
dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile
addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts

These configuration files should not be edited manually, but with:

   virsh net-edit <virtual-network-name>

The virtual network switch can operate in several modes, described below, and by default it operates in NAT mode:

• NAT mode
• Routed mode
• Bridged mode
• Isolated mode

Virtual Network Modes

NAT Mode

The NAT mode is the default mode in which libvirt virtual network switch operates, without additional configuration.

All guests have direct connectivity to each other and to the virtualization host. libvirt network filtering and guest operating system iptables rules apply. The guests can access external networks by network address translation, subject to the host system's firewall rules. The network address translation function is implemented as IP masquerading, using iptables rules. Connected guests will use the virtualization host physical machine IP address for communication with external networks (it is possible that the virtualization host IP address is unroutable as well, if the virtualization host sits in its own NAT domain). By default, without additional configuration, external hosts cannot initiate connections to the guest virtual machines. If incoming connections need to be handled by guests, this is possible by setting up libvirt's "hook" script for qemu to install the necessary iptables rules to forward incoming connections. More details: https://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections

Routed Mode

The libvirt virtual network switch can be configured to run in routed mode. In routed mode, the switch connects to the physical LAN the virtualization host is attached to, without the intermediation of a NAT module. All the guest virtual machines are in the same subnet, routed through the virtual switch. External traffic may reach the guest only if additional routing entires are added. The routed mode operates at Layer 3 of the OSI networking model.

Use Cases:

• Implementation of a DMZ (TODO): https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/sect-Virtual_Networking-Examples_of_common_scenarios.html#sect-Examples_of_common_scenarios-Routed_mode
• Virtual Server Hosting: (TODO): https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/sect-Virtual_Networking-Examples_of_common_scenarios.html#sect-Examples_of_common_scenarios-Routed_mode

Bridged Mode

In bridged mode ("shared physical device mode"), the guests are connected to a bridge device that is also connected directly to a physical ethernet device connected to the local ethernet. This makes the guests directly visible on the physical network, as being part of the same subnet as the virtualization host, and thus enables incoming connections, but does not require any extra routing table entries. All other physical machines on the same physical network are aware of the virtual machines at MAC address level, and they can access them. Bridging operates on Layer 2 of the OSI networking model.

It is possible to use multiple physical network interfaces on the virtualization host by joining them together with a bond. The bond is then added to the bridge. For more details, see network bonding.

Use cases for bridged mode:

• When the guest virtual machines need to be deployed in an existing network alongside virtualization hosts and other physical machines, and they should appear no different to the end user.
• When the guests need to be deployed without making any changes to the existing physical network configuration settings.
• When the guests must be easily accessible from an existing physical network and they must use already deployed services.
• When the guests must be connected to an existing network where VLANs are used.

Isolated Mode

If the libvirt virtual network switch is configured to run in isolated mode, the guest virtual machines can communicate with each other and with the virtualization host, but the traffic will not pass outside of the virtualization host, not can they receive external traffic. The guests still receive their addresses via DHCP from dnsmasq, and send their DNS queries to the same dnsmasq instance, which may resolve external names, but the resolved IP addresses won't be accessible.

Guest Configuration for Virtual Network Mode

<devices>
  ...
  <interface type='network'>
    <source network='default'/>
  </interface>
  ...
</devices>

Guests Attaches Directly to a Virtualization Host Network Device

macvtap Driver

The macvtap driver can be used to attach the guest's NIC directly to a specified physical interface of the host machine. This is not to be confused with PCI device assignment (also known as passthrough). Macvtap connection has the following modes, each with different benefits and usecases:

...

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/sect-virtual_networking-directly_attaching_to_physical_interface

Directly Attaching a Guest Virtual Machine to a Physical Network Interface

PCI device assignment

PCI device assignment is also known as "passthrough".

• TODO: https://wiki.libvirt.org/page/Networking#PCI_Passthrough_of_host_network_devices
• TODO: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/chap-Guest_virtual_machine_device_configuration

libvirt Network Filtering

• TODO: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/sect-Virtual_Networking-Applying_network_filtering.html