Attaching a Guest Directly to a Virtualization Host Network Interface with a macvtap Driver
External
Internal
Overview
This procedure assumes the guest wants to have two network interfaces: one that connects the guest to an internal virtual network, used by guests to communicate with each other (eth0), and the second network interface, enabled by a macvtap driver, attached directly to a visualization host network interface (ens8).
The procedure leaves the network interface that will be used for direct attachment unconfigured on the virtualization host and declares a network interface of type "direct" in the guest configuration. After assignment, and after the guest boots, the corresponding interface shows up on host as follows:
3: em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 18:66:da:9f:96:d7 brd ff:ff:ff:ff:ff:ff
...
11: macvtap0@em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state LOWERLAYERDOWN qlen 500
link/ether 52:54:00:02:72:ed brd ff:ff:ff:ff:ff:ff
The same interface shows up as ens8 on the guest:
2: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:02:72:ed brd ff:ff:ff:ff:ff:ff
Note that the virtualization host macvtap interface (macvtap0@em2) and the guest interface have the same MAC address.
Procedure
Configure the Physical Network Interface on the Virtualization Host
Leave the network interface unconfigured on the virtualization host. The corresponding configuration file /etc/sysconfig/network-scripts/ifcfg-em3 should show the following:
NAME=em3
DEVICE=em3
TYPE=Ethernet
ONBOOT=no
BOOTPROTO=none
DEFROUTE=no
PEERDNS=no
PEERROUTES=no
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
After reboot, it should look like this:
4: em3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 18:66:da:9f:96:d8 brd ff:ff:ff:ff:ff:ff
Configure the Virtual Device on Guest
With the guest down, assign it to the guest in configuration with virsh edit or in the corresponding XML configuration file as follows:
...
<interface type='direct'>
<source dev='em3' mode='private'/>
</interface>
...
where "em3" is the virtualization host network interface to be directly exposed to the guest.
Multiple guests can attach to the same virtualization host physical interface, and in "private" mode, they will be isolated from each other. Each guest can be individually configured with IP addresses that make sense on the network the physical address is connected to, while the corresponding network interface on the virtualization host will stay unconfigured. Other modes ("VEPA", "bridge", etc.) are available.
Note that after creating the domain based on the XML configuration, libvirt will update the stored configuration as follows:
...
<interface type='direct'>
<mac address='...'/>
<source dev='em3' mode='private'/>
<model type='rtl8139'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</interface>
...
After guest reboot, the MAC address reported by the configuration must corresponds to the MAC address of the virtualization host macvtap...@em3 interface on the virtualization host, and it must be different from the MAC address of the virtualization host em3 interface. Optionally, a specific MAC address could be configured on the libvirt guest interface, by specifying
<interface type='direct'>
...
<mac address='...'/>
...
</interface>
...
in the guest configuration.
The new network interface shows up on the guest, alongside the default network interface eth1:
ip addr
...
2: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:02:72:ed brd ff:ff:ff:ff:ff:ff
...
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:e4:e6:8a brd ff:ff:ff:ff:ff:ff
The MAC address reported by the guest mavtap ens8 interface must be the same as the MAC address of the corresponding mavtap...@em3 on the virtualization host.
Primary Interface Name Instability
A certain primary interface name instability has been observed after adding the second virtual network interface in the domain definition file. The name of the primary network interface would be assigned non-deterministically to "eth0" and "eth1", respectively, upon successive reboots. An attempt to configure "device" in the domain name interface definition was ineffective. The issue was worked around - without a good explanation of why it is happening - by adding both /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1. As a side effect of the instability, NetworkManager was not able to properly detect the interfaces at boot, so it was disabled: Disable NetworkManager and Configure the Interfaces Manually.
Configure the Network Interface on Guest
eth0 Manual Configuration
Make sure that /etc/sysconfig/network-scripts/ifcfg-eth0 contains:
DEFROUTE=no
The default route will be provided by the directly attached interface.
ens8 Manual Configuration
After the first boot, figure out the name of the new network interface and add a /etc/sysconfig/network-scripts/ifcfg-<interface-name> configuration file, with a content similar to the one shown below.
Make sure to configure:
DEFROUTE=yes
Also make sure UUID is unique, you can generate a new UUID with uuidgen.
NAME=ens8
DEVICE=ens8
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=none
IPADDR=104.50.201.84
PREFIX=29
GATEWAY=104.50.201.86
DEFROUTE=yes
PEERDNS=no
PEERROUTES=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
UUID=800628FE-6265-4902-80A6-4FC0D018D305
DNS1=8.8.8.8
For more details on how to configure network interfaces, see
Reboot the second time, both interfaces should be operational.
Verification
After reboot, the guest must be accessible over ssh from an external network.
The routing table should be similar to:
Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 104.50.201.86 0.0.0.0 UG 0 0 0 ens8 104.50.201.80 0.0.0.0 255.255.255.248 U 0 0 0 ens8 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ens8 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1