Attaching a Guest Directly to a Virtualization Host Network Interface with a macvtap Driver

External

• https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/sect-Virtual_Networking-Directly_attaching_to_physical_interface.html

Internal

• Linux Virtualization Networking Concepts
• Linux Virtualization Operations

Overview

This procedure assumes the guest wants to have two network interfaces: one that connects the guest to an internal virtual network, used by guests to communicate with each other (eth0), and the second network interface, enabled by a macvtap driver, attached directly to a visualization host network interface.

The procedure leaves the network interface that will be used for direct attachment unconfigured on the virtualization host and declares a network interface of type "direct" in the guest configuration. After assignment, and after the guest boots, the corresponding interface shows up on host as follows:

3: em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 18:66:da:9f:96:d7 brd ff:ff:ff:ff:ff:ff
...
11: macvtap0@em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state LOWERLAYERDOWN qlen 500
    link/ether 52:54:00:02:72:ed brd ff:ff:ff:ff:ff:ff

Procedure

Configure the Virtual Device on Guest

Leave the network interface unconfigured on the virtualization host.

Assign it to the guest in configuration with virsh edit or in the corresponding XML configuration file as follows:

...
<interface type='direct'>
   <source dev='em2' mode='private'/>
</interface>
...

where "em2" is the virtualization host network interface to be directly exposed to the guest.

Note that after creating the domain based on the XML configuration, libvirt will update the stored configuration as follows:

...
<interface type='direct'>
   <mac address='...'/>
   <source dev='em2' mode='private'/>
   <model type='rtl8139'/>
   <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</interface>
...

Note that the MAC address generated in the configuration is different from the MAC address returned by ip addr on the virtualization host.

The new network interface shows up on the guest, alongside the default network interface eth1:

ip addr
...
2: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:02:72:ed brd ff:ff:ff:ff:ff:ff
...
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:e4:e6:8a brd ff:ff:ff:ff:ff:ff

Primary Interface Name Instability

A certain primary interface name instability has been observed after adding the second virtual network interface in the domain definition file. The name of the primary network interface would be assigned non-deterministically to "eth0" and "eth1", respectively, upon successive reboots. An attempt to configure "device" in the domain name interface definition was ineffective. The issue was worked around - without a good explanation of why it is happening - by adding both /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1. As a side effect of the instability, NetworkManager was not able to properly detect the interfaces at boot, so it was disabled: Disable NetworkManager and Configure the Interfaces Manually.

Configure the Network Interface on Guest

eth0 Manual Configuration

Make sure that /etc/sysconfig/network-scripts/ifcfg-eth0 contains:

DEFROUTE=no

The default route will be provided by the directly attached interface.

ens8 Manual Configuration

After the first boot, figure out the name of the new network interface and add a /etc/sysconfig/network-scripts/ifcfg-<interface-name> configuration file, with a content similar to the one shown below.

Make sure to configure:

DEFROUTE=yes

Also make sure UUID is unique, you can generate a new UUID with uuidgen.

NAME=ens8
DEVICE=ens8
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=none
IPADDR=104.50.201.84
PREFIX=29
GATEWAY=104.50.201.86
DEFROUTE=yes
PEERDNS=no
PEERROUTES=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
UUID=800628FE-6265-4902-80A6-4FC0D018D305
DNS1=8.8.8.8

For more details on how to configure network interfaces, see

Configuring a Network Interface

Reboot the second time, both interfaces should be operational.

Verification

After reboot, the guest must be accessible over ssh from an external network.

The routing table should be similar to:

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         104.50.201.86   0.0.0.0         UG        0 0          0 ens8
104.50.201.80   0.0.0.0         255.255.255.248 U         0 0          0 ens8
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 ens8
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth1
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 eth1