Attaching a Guest Directly to a Virtualization Host Network Interface with a macvtap Driver: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(38 intermediate revisions by the same user not shown)
Line 11: Line 11:
=Overview=
=Overview=


This procedure assumes the guest wants to have at least two kinds of network interfaces: one that connects the guest to an [[KVM_Virtual_Networking_Concepts#Virtual_Network| internal virtual network]], used by guests to communicate with each other (<code>eth0</code>), and the others, enabled by a [[KVM_Virtual_Networking_Concepts#macvtap_Driver|macvtap driver]], attached directly to the visualization host network interfaces (<code>ens8</code>, <code>ens9</code>). These virtualization host interfaces can be connected to different networks. Note that at least two of these virtual network interfaces are necessary when the host services public addresses: one of the interfaces will be used to service external traffic, and the other, internal LAN (192.168.1.*) traffic.
This procedure assumes the guest wants to have at least two kinds of network interfaces: one (<code>eth0</code>) connecting the guest to an [[KVM_Virtual_Networking_Concepts#Virtual_Network| internal virtual network]] that exists inside the virtualization host and it is used by guests to communicate with each other, and the others (<code>ens8</code>, <code>ens9</code>), enabled by a [[KVM_Virtual_Networking_Concepts#macvtap_Driver|macvtap driver]], attached directly to the visualization host network interfaces. These interfaces can be used by the guest to connect to different networks. Usually, there is a network interface that routes traffic to the LAN (192.168.1.*) traffic., and the other with a publicly routable IP address that services external traffic.


The procedure is written for two of such network interfaces. The first one is assumed to connect to the internal LAN, and the second one to the public network.
This procedure is written for two such network interfaces. The first one is assumed to connect the guest to the internal LAN, and the second one to a public network.


The procedure leaves the virtualization host network interfaces that will be used for direct attachment '''unconfigured''' (the hardware will be used directly by the guest network stack) and declares a network interface of type "direct" in the guest configuration.  
The procedure leaves the virtualization host network interfaces that will be used for direct attachment '''unconfigured''' (the hardware will be used directly by the guest network stack) and declares a network interface of type "direct" in the guest configuration.  
Line 27: Line 27:
         valid_lft forever preferred_lft forever
         valid_lft forever preferred_lft forever
  <font color=SlateGray># this macvtap interface corresponds to the ens9 (internet facing) interface on guest. The common element is the MAC address: 52:54:00:43:e0:2f</font>
  <font color=SlateGray># this macvtap interface corresponds to the ens9 (internet facing) interface on guest. The common element is the MAC address: 52:54:00:43:e0:2f</font>
  23: macvtap5@em3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state LOWERLAYERDOWN qlen 500
  23: macvtap5@em3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
     link/ether 52:54:00:43:e0:2f brd ff:ff:ff:ff:ff:ff
     link/ether 52:54:00:bc:e5:36 brd ff:ff:ff:ff:ff:ff
     [...]
     inet6 fe80::5054:ff:febc:e536/64 scope link
        valid_lft forever preferred_lft forever
</font>
</font>


The macvtap label index may vary, even between boots. <font color=darkgray>It is probably possible to request specific labels when configuring the interface on the guest.</font>
The macvtap label index may vary, even between boots. <font color=darkkhaki>How can we request specific "macvtapX@" labels, deterministically? So far, these were generated automatically.</font>


The corresponding guest interfaces are <code>ens8</code> and <code>ens9</code> (or similar):
The corresponding guest interfaces are <code>ens8</code> and <code>ens9</code> (or similar):
Line 58: Line 59:
==Configure the Physical Network Interface on the Virtualization Host==
==Configure the Physical Network Interface on the Virtualization Host==


In case of the virtualization host interface that connects the virtualization host to the local network, that is already configured, similar to:
In case of the virtualization host interface that connects the virtualization host to the local network (<code>em1</code>), it is surely already configured. The content of <code>/etc/sysconfig/network-scripts/ifcfg-em1</code> should be similar to:


<syntaxhighlight lang='text'>
<syntaxhighlight lang='bash'>
BOOTPROTO="none"
BOOTPROTO="none"
DEFROUTE="yes"
DEFROUTE="yes"
Line 83: Line 84:
</syntaxhighlight>
</syntaxhighlight>


Leave the network interface, in this case <code>em1</code>, '''unconfigured''' on the virtualization host. The corresponding configuration file <code>/etc/sysconfig/network-scripts/ifcfg-em1</code> should show the following (only essential configuration is shown):
In case of the virtualization host network interface that gives access to internet (<code>em3</code>), leave it '''unconfigured''' on the virtualization host. The corresponding configuration file <code>/etc/sysconfig/network-scripts/ifcfg-em3</code> should show the following (only essential configuration is shown):


<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
NAME=em2
BOOTPROTO="none"
DEVICE=em2
DEFROUTE="no" # Does not matter, will not be used by the virtualization host
TYPE=Ethernet
DEVICE="em3"
ONBOOT=no # No network interface should be started at boot, the hardware will be shared by guests
IPV4_FAILURE_FATAL="no"
BOOTPROTO=no
IPV6_AUTOCONF="no"
DEFROUTE=no # Does not matter, will not be used by the virtualization host
IPV6_DEFROUTE="no"
PEERDNS=no
IPV6_FAILURE_FATAL="no"
PEERROUTES=no
IPV6INIT="no"
IPV4_FAILURE_FATAL=no
IPV6_PEERDNS="no"
IPV6INIT=no
IPV6_PEERROUTES="no"
IPV6_AUTOCONF=no
NAME="em3"
IPV6_DEFROUTE=no
ONBOOT="no" # No network interface should be started at boot, the hardware will be shared by guests
IPV6_PEERDNS=no
PEERDNS="no"
IPV6_PEERROUTES=no
PEERROUTES="no"
IPV6_FAILURE_FATAL=no
TYPE="Ethernet"
UUID=85e14c6b-f9d3-49a0-b566-f122d47001c6
UUID="85e14c6b-f9d3-49a0-b566-f122d47001c6"
</syntaxhighlight>
</syntaxhighlight>


After reboot, it should look like this:
After reboot, the virtualization host interfaces look similar to:
<syntaxhighlight lang='text'>
<font size=-2>
3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 18:66:da:9f:96:d7 brd ff:ff:ff:ff:ff:ff
    link/ether 18:66:da:9f:96:d6 brd ff:ff:ff:ff:ff:ff
    inet6 2600:1700:dc41:3940:1a66:daff:fe9f:96d7/64 scope global mngtmpaddr dynamic
    inet 192.168.1.13/24 brd 192.168.1.255 scope global em1
      valid_lft 3543sec preferred_lft 3543sec
        valid_lft forever preferred_lft forever
    inet6 fe80::1a66:daff:fe9f:96d7/64 scope link
    inet6 fe80::1a66:daff:fe9f:96d6/64 scope link
      valid_lft forever preferred_lft forever
        valid_lft forever preferred_lft forever
</syntaxhighlight>
[...]
 
4: em3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
The example above applies to a network interface that is left unconfigured. It is also possible to use an interface that is configured and used by the virtualization host for its own purposes:
    link/ether 18:66:da:9f:96:d8 brd ff:ff:ff:ff:ff:ff
<syntaxhighlight lang='text'>
    inet6 2600:1700:543c:b010:1a66:daff:fe9f:96d8/64 scope global mngtmpaddr dynamic
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
        valid_lft 2591942sec preferred_lft 604742sec
    link/ether 18:66:da:9f:96:d6 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::1a66:daff:fe9f:96d8/64 scope link
    inet 192.168.1.13/24 brd 192.168.1.255 scope global em1
        valid_lft forever preferred_lft forever
      valid_lft forever preferred_lft forever
</font>
    inet6 fe80::1a66:daff:fe9f:96d6/64 scope link
      valid_lft forever preferred_lft forever
</syntaxhighlight>


==Configure the Virtual Device on Guest==
==Configure the Virtual Device on Guest==
Line 129: Line 127:
<syntaxhighlight lang='xml'>
<syntaxhighlight lang='xml'>
...
...
<interface type='direct'>
<domain type='kvm' id='9'>
  <source dev='em2' mode='private'/>
  [...]
</interface>
  <devices>
 
    [...]
<!-- More than one interface can be configured in one step -->
    <interface type='direct'>
<interface type='direct'>
      <source dev='em1' mode='private'/>
  <source dev='em1' mode='private'/>
    </interface>
</interface>
    <!-- More than one interface can be configured in one step -->
...
    <interface type='direct'>
      <source dev='em3' mode='private'/>
    </interface>
  </devices>
</domain>
</syntaxhighlight>
</syntaxhighlight>


where "em2" is the virtualization host network interface to be directly exposed to the guest.
This is the configuration step where the virtualization host network interfaces are chosen. Pick one that connects to the LAN, and another one that connects to a gateway device. <code>&lt;source dev='emX' ... /&gt;</code> is the configuration line that selects the virtualization host interface to attach to. Multiple guests and multiple interfaces on the same guest may attach to the same virtualization host interface.


{{Note|Multiple guests can attach to the same virtualization host physical interface, and in "[[KVM_Virtual_Networking_Concepts#private|private]]" mode, they will be isolated from each other. Each guest can be individually configured with IP addresses that make sense on the network the physical address is connected to, while the corresponding network interface on the virtualization host will stay unconfigured. Other modes ("[[KVM_Virtual_Networking_Concepts#VEPA|VEPA]]", "[[KVM_Virtual_Networking_Concepts#bridge|bridge]]", etc.) are available.}}
{{Note|Multiple guests can attach to the same virtualization host physical interface, and in "[[KVM_Virtual_Networking_Concepts#private|private]]" mode, they will be isolated from each other. Each guest can be individually configured with IP addresses that make sense on the network the physical address is connected to, while the corresponding network interface on the virtualization host will stay unconfigured. Other modes ("[[KVM_Virtual_Networking_Concepts#VEPA|VEPA]]", "[[KVM_Virtual_Networking_Concepts#bridge|bridge]]", etc.) are available.}}
Line 150: Line 152:
</syntaxhighlight>
</syntaxhighlight>
<syntaxhighlight lang='xml'>
<syntaxhighlight lang='xml'>
...
<domain type='kvm' id='9'>
<interface type='direct'>
  [...]
  <mac address='52:54:00:5d:9c:33'/>
  <devices>
  <source dev='em2' mode='private'/>
    [...]
  <target dev='macvtap1'/>
    <interface type='direct'>
  <model type='rtl8139'/>
      <mac address='52:54:00:4d:8e:35'/>
  <alias name='net2'/>
      <source dev='em1' mode='private'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
      <target dev='macvtap4'/>
</interface>
      <model type='rtl8139'/>
...
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='direct'>
      <mac address='52:54:00:43:e0:2f'/>
      <source dev='em3' mode='private'/>
      <target dev='macvtap5'/>
      <model type='rtl8139'/>
      <alias name='net2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
  </devices>
</domain>
</syntaxhighlight>
</syntaxhighlight>


After guest reboot, the MAC address reported by the configuration must corresponds to the MAC address of the virtualization host macvtap1@em2 interface on the virtualization host, and it must be different from the MAC address of the virtualization host em2 interface.  
After guest reboot, the MAC address reported by the configuration, on the guest, must corresponds to the MAC address of the corresponding virtualization host <code>macvtap*@em*</code> interfaces.


Optionally, a specific MAC address could be configured on the <code>libvirt</code> guest interface:
Optionally, a specific MAC address could be configured on the <code>libvirt</code> guest interface:
Line 176: Line 190:


in the guest configuration.
in the guest configuration.
The new network interface, in this case ens9, shows up on the guest, alongside the default network interface eth0:
<syntaxhighlight lang='text'>
<syntaxhighlight lang='bash'>
ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
[...]
3: ens9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:5d:9c:33 brd ff:ff:ff:ff:ff:ff
    inet 104.50.201.84/29 brd 104.50.201.87 scope global noprefixroute ens9
4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:b4:87:0f brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.249/24 brd 192.168.122.255 scope global noprefixroute dynamic eth0
</syntaxhighlight>
The MAC address reported by the guest mavtap ens9 interface must be the same as the MAC address of the corresponding mavtap1@em2 on the virtualization host.


==Primary Interface Name Instability==
==Primary Interface Name Instability==
Line 202: Line 199:
===eth0 Manual Configuration===
===eth0 Manual Configuration===


⚠️ This is important, if the eth0 is not configured as shown below, the new network interface will not work correctly.
⚠️ This is important, if the <code>eth0</code> is not configured as shown below, the new network interface will not work correctly.


Make sure that <code>/etc/sysconfig/network-scripts/ifcfg-eth0</code> contains:
Make sure that <code>/etc/sysconfig/network-scripts/ifcfg-eth0</code> contains:
Line 211: Line 208:
</syntaxhighlight>
</syntaxhighlight>


The default route will be provided by the macvtap interface, so [[Linux_7_Configuring_a_Network_Interface#DEFROUTE|DEFROUTE]] should be set to "no". Also, we don't want DHCP to modify <code>/etc/resolv.conf</code> so [[Linux_7_Configuring_a_Network_Interface#PEERDNS|PEERDNS]] is set to false.
The default route will be provided by the '''public''' macvtap interface, so [[Linux_7_Configuring_a_Network_Interface#DEFROUTE|DEFROUTE]] should be set to "no". Also, we don't want DHCP to modify <code>/etc/resolv.conf</code> so [[Linux_7_Configuring_a_Network_Interface#PEERDNS|PEERDNS]] is set to false.


===ens9 (macvtap interface) Manual Configuration===
===ens8 (macvtap interface that connects to the LAN) Manual Configuration===


⚠️ This is important, if the macvtap interface is not configured as shown below, it will not work correctly.
⚠️ This is important, if the macvtap interface is not configured as shown below, it will not work correctly.
Line 221: Line 218:
Update <code>/etc/sysconfig/network-scripts/ifcfg-ens8</code> as follows:
Update <code>/etc/sysconfig/network-scripts/ifcfg-ens8</code> as follows:


Make sure to configure:
Make sure to configure <code>DEFROUTE="no"</code>. For external access to work, the default route should use the public interface.


<syntaxhighlight lang='text'>
<syntaxhighlight lang='bash'>
DEFROUTE=yes # macvtap interface will provide the default route
DEFROUTE="no" # the public macvtap interface will provide the default route, not this one
</syntaxhighlight>
</syntaxhighlight>


Line 231: Line 228:
The configuration should be similar to:
The configuration should be similar to:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
NAME=ens9
BOOTPROTO="none"
DEVICE=ens9
DEFROUTE=no # the public macvtap interface will provide the default route, not this one
TYPE=Ethernet
DEVICE="ens8"
ONBOOT=yes
DNS1="8.8.8.8"
BOOTPROTO=none
GATEWAY="192.168.1.1"
IPADDR=104.50.201.84
IPADDR="192.168.1.15"
NETMASK=255.255.255.248
IPV4_FAILURE_FATAL="yes"
PREFIX=29
IPV6_AUTOCONF="no"
GATEWAY=104.50.201.86
IPV6_DEFROUTE="no"
DEFROUTE=yes # macvtap interface will provide the default route
IPV6_FAILURE_FATAL="no"
PEERROUTES=no
IPV6INIT="no"
PEERDNS=no
IPV6_PEERDNS="no"
DNS1=8.8.8.8
IPV6_PEERROUTES="no"
IPV4_FAILURE_FATAL=yes
NAME="ens8"
IPV6INIT=no
NETMASK="255.255.255.0"
IPV6_AUTOCONF=no
ONBOOT="yes"
IPV6_DEFROUTE=no
PEERDNS="no"
IPV6_PEERDNS=no
PEERROUTES="no"
IPV6_PEERROUTES=no
PREFIX="24"
IPV6_FAILURE_FATAL=no
TYPE="Ethernet"
UUID=010B8B2C-9809-48E1-B14F-0BF224A8EFB2
UUID=adcf7be8-0dd7-46f5-b572-c6d273e23d87
</syntaxhighlight>
</syntaxhighlight>
For more details on how to configure network interfaces, see {{Internal|Linux 7 Configuring a Network Interface|Configuring a Network Interface}}
For more details on how to configure network interfaces, see {{Internal|Linux 7 Configuring a Network Interface|Configuring a Network Interface}}


===Second macvtap interface Manual Configuration===
===Second, public macvtap (ens9) interface Manual Configuration===
 
A second macvtap interface connects the guest to public internet. The configuration should be similar to the example provided below, and written in <code>/etc/sysconfig/network-scripts/ifcfg-ens9</code>. <code>"DEFROUTE=yes"</code> is essential.
 
⚠️ Important: The routable IP address used below must not be used by another guests.


In case a second macvtap interface connects the guest to an internal network, the configuration should be similar to the example provided below. Note the "DEFROUTE=no" configuration.
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
NAME=ens8
BOOTPROTO="none"
DEVICE=ens8
DEFROUTE="yes" # essential
TYPE=Ethernet
DEVICE="ens9"
ONBOOT=yes
DNS1="8.8.8.8"
BOOTPROTO=none
GATEWAY="104.50.201.86"
IPADDR=192.168.1.15
IPADDR="104.50.201.85" # must not conflict with other guests' public address
PREFIX=24
IPV4_FAILURE_FATAL="yes"
GATEWAY=192.168.1.1
IPV6_AUTOCONF="no"
DEFROUTE=no
IPV6_DEFROUTE="no"
PEERROUTES=no
IPV6_FAILURE_FATAL="no"
PEERDNS=no
IPV6INIT="no"
DNS1=8.8.8.8
IPV6_PEERDNS="no"
IPV4_FAILURE_FATAL=yes
IPV6_PEERROUTES="no"
IPV6INIT=no
NAME="ens9"
IPV6_AUTOCONF=no
NETMASK="255.255.255.248"
IPV6_DEFROUTE=no
ONBOOT="yes"
IPV6_PEERDNS=no
PEERDNS="no"
IPV6_PEERROUTES=no
PEERROUTES="no"
IPV6_FAILURE_FATAL=no
PREFIX="29"
UUID=7af78337-479d-47e2-8b16-a04dc42227c8
TYPE="Ethernet"
UUID="30C65EA0-1EC3-46C6-8AC1-202D83102B35"
</syntaxhighlight>
</syntaxhighlight>


Line 288: Line 290:


The routing table should be similar to:
The routing table should be similar to:
<syntaxhighlight lang='text'>
<font size=-1>
Destination    Gateway        Genmask        Flags  MSS Window  irtt Iface
Destination    Gateway        Genmask        Flags  MSS Window  irtt Iface
0.0.0.0        104.50.201.86  0.0.0.0        UG        0 0          0 ens9
0.0.0.0        104.50.201.86  0.0.0.0        UG        0 0          0 ens9
104.50.201.80  0.0.0.0        255.255.255.248 U        0 0          0 ens9
104.50.201.80  0.0.0.0        255.255.255.248 U        0 0          0 ens9
192.168.1.0    0.0.0.0        255.255.255.0  U        0 0          0 ens8
169.254.0.0    0.0.0.0        255.255.0.0    U        0 0          0 ens8
192.168.122.0  0.0.0.0        255.255.255.0  U        0 0          0 eth0
169.254.0.0    0.0.0.0        255.255.0.0    U        0 0          0 ens9
</syntaxhighlight>
169.254.0.0    0.0.0.0        255.255.0.0    U        0 0          0 eth0
192.168.1.0    0.0.0.0        255.255.255.0  U        0 0          0 ens8
192.168.122.0  0.0.0.0        255.255.255.0  U        0 0          0 eth0
</font>

Latest revision as of 03:45, 6 August 2023

External

Internal

Overview

This procedure assumes the guest wants to have at least two kinds of network interfaces: one (eth0) connecting the guest to an internal virtual network that exists inside the virtualization host and it is used by guests to communicate with each other, and the others (ens8, ens9), enabled by a macvtap driver, attached directly to the visualization host network interfaces. These interfaces can be used by the guest to connect to different networks. Usually, there is a network interface that routes traffic to the LAN (192.168.1.*) traffic., and the other with a publicly routable IP address that services external traffic.

This procedure is written for two such network interfaces. The first one is assumed to connect the guest to the internal LAN, and the second one to a public network.

The procedure leaves the virtualization host network interfaces that will be used for direct attachment unconfigured (the hardware will be used directly by the guest network stack) and declares a network interface of type "direct" in the guest configuration.

After assignment, and after the guest boots, the corresponding virtualization host interfaces show up as follows: 

[...]
# this macvtap interface corresponds to the ens8 (local network) interface on guest. The common element is the MAC address: 52:54:00:4d:8e:35
22: macvtap4@em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
    link/ether 52:54:00:4d:8e:35 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fe4d:8e35/64 scope link
       valid_lft forever preferred_lft forever
# this macvtap interface corresponds to the ens9 (internet facing) interface on guest. The common element is the MAC address: 52:54:00:43:e0:2f
23: macvtap5@em3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
    link/ether 52:54:00:bc:e5:36 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:febc:e536/64 scope link
       valid_lft forever preferred_lft forever

The macvtap label index may vary, even between boots. How can we request specific "macvtapX@" labels, deterministically? So far, these were generated automatically.

The corresponding guest interfaces are ens8 and ens9 (or similar): 

[...]
# this guest interface corresponds to macvtap4@em1. The common element is the MAC address: 52:54:00:4d:8e:35
2: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:4d:8e:35 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.15/24 brd 192.168.1.255 scope global ens8
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe4d:8e35/64 scope link
       valid_lft forever preferred_lft forever
# this guest interface corresponds to macvtap5@em3. The common element is the MAC address:52:54:00:43:e0:2f
3: ens9: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 52:54:00:43:e0:2f brd ff:ff:ff:ff:ff:ff
    [...]

The virtualization host macvtap interface and its corresponding guest interface have the same MAC address.

Multiple independent macvtap interfaces can be set on the same virtualization host physical network interface, sharing the hardware. They can be configured independently with different IP addresses. More details are available in the Configure the Virtual Device on Guest section, below.

Procedure

Configure the Physical Network Interface on the Virtualization Host

In case of the virtualization host interface that connects the virtualization host to the local network (em1), it is surely already configured. The content of /etc/sysconfig/network-scripts/ifcfg-em1 should be similar to: 

BOOTPROTO="none"
DEFROUTE="yes"
DEVICE="em1"
DNS1="192.168.1.1"
DNS2="8.8.8.8"
DOMAIN="local"
GATEWAY="192.168.1.1"
IPADDR="192.168.1.13"
IPV4_FAILURE_FATAL="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6INIT="no"
IPV6_PEERDNS="yes"
IPV6_PEERROUTES="yes" 
NAME="em1"
ONBOOT="yes"
PREFIX="24"
TYPE="Ethernet"
UUID="75f88553-aa66-4783-aa47-8430c3757fa5"

In case of the virtualization host network interface that gives access to internet (em3), leave it unconfigured on the virtualization host. The corresponding configuration file /etc/sysconfig/network-scripts/ifcfg-em3 should show the following (only essential configuration is shown): 

BOOTPROTO="none"
DEFROUTE="no" # Does not matter, will not be used by the virtualization host
DEVICE="em3"
IPV4_FAILURE_FATAL="no"
IPV6_AUTOCONF="no"
IPV6_DEFROUTE="no"
IPV6_FAILURE_FATAL="no"
IPV6INIT="no"
IPV6_PEERDNS="no"
IPV6_PEERROUTES="no"
NAME="em3"
ONBOOT="no" # No network interface should be started at boot, the hardware will be shared by guests
PEERDNS="no"
PEERROUTES="no"
TYPE="Ethernet"
UUID="85e14c6b-f9d3-49a0-b566-f122d47001c6"

After reboot, the virtualization host interfaces look similar to: 

2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 18:66:da:9f:96:d6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.13/24 brd 192.168.1.255 scope global em1
       valid_lft forever preferred_lft forever
    inet6 fe80::1a66:daff:fe9f:96d6/64 scope link
       valid_lft forever preferred_lft forever
[...]
4: em3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 18:66:da:9f:96:d8 brd ff:ff:ff:ff:ff:ff
    inet6 2600:1700:543c:b010:1a66:daff:fe9f:96d8/64 scope global mngtmpaddr dynamic
       valid_lft 2591942sec preferred_lft 604742sec
    inet6 fe80::1a66:daff:fe9f:96d8/64 scope link
       valid_lft forever preferred_lft forever

Configure the Virtual Device on Guest

Follow the guest network reconfiguration procedure and update the network configuration as follows: 

...
<domain type='kvm' id='9'>
  [...]
  <devices>
    [...]
    <interface type='direct'>
      <source dev='em1' mode='private'/>
    </interface>
    <!-- More than one interface can be configured in one step -->
    <interface type='direct'>
      <source dev='em3' mode='private'/>
    </interface>
  </devices>
</domain>

This is the configuration step where the virtualization host network interfaces are chosen. Pick one that connects to the LAN, and another one that connects to a gateway device. <source dev='emX' ... /> is the configuration line that selects the virtualization host interface to attach to. Multiple guests and multiple interfaces on the same guest may attach to the same virtualization host interface.


Multiple guests can attach to the same virtualization host physical interface, and in "private" mode, they will be isolated from each other. Each guest can be individually configured with IP addresses that make sense on the network the physical address is connected to, while the corresponding network interface on the virtualization host will stay unconfigured. Other modes ("VEPA", "bridge", etc.) are available.

Note that after creating the domain based on the XML configuration, libvirt will update the stored configuration as follows: 

virsh dumpxml <guest-name>

<domain type='kvm' id='9'>
  [...]
  <devices>
    [...]
    <interface type='direct'>
      <mac address='52:54:00:4d:8e:35'/>
      <source dev='em1' mode='private'/>
      <target dev='macvtap4'/>
      <model type='rtl8139'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='direct'>
      <mac address='52:54:00:43:e0:2f'/>
      <source dev='em3' mode='private'/>
      <target dev='macvtap5'/>
      <model type='rtl8139'/>
      <alias name='net2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
  </devices>
</domain>

After guest reboot, the MAC address reported by the configuration, on the guest, must corresponds to the MAC address of the corresponding virtualization host macvtap*@em* interfaces.

Optionally, a specific MAC address could be configured on the libvirt guest interface: 

<interface type='direct'>
   ...
   <mac address='...'/>
   ...
</interface>
...

in the guest configuration.

Primary Interface Name Instability


A certain primary interface name instability has been observed after adding the second virtual network interface in the domain definition file. The name of the primary network interface would be assigned non-deterministically to "eth0" and "eth1", respectively, upon successive reboots. An attempt to configure "device" in the domain name interface definition was ineffective. The issue was worked around - without a good explanation of why it is happening - by adding both /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1. As a side effect of the instability, NetworkManager was not able to properly detect the interfaces at boot, so it was disabled: Disable NetworkManager and Configure the Interfaces Manually.

Configure the Network Interface on Guest

eth0 Manual Configuration

⚠️ This is important, if the eth0 is not configured as shown below, the new network interface will not work correctly.

Make sure that /etc/sysconfig/network-scripts/ifcfg-eth0 contains: 

DEFROUTE=no # The default route will be provided by the macvtap interface.
IPV6_DEFROUTE=no
PEERDNS=no # We do not want DHCP to modify /etc/resolv.conf

The default route will be provided by the public macvtap interface, so DEFROUTE should be set to "no". Also, we don't want DHCP to modify /etc/resolv.conf so PEERDNS is set to false.

ens8 (macvtap interface that connects to the LAN) Manual Configuration

⚠️ This is important, if the macvtap interface is not configured as shown below, it will not work correctly.

Verify that the MAC address of the ens8 interface on guest matches the MAC address of the macvtap4@em1 interface on the virtualization host.

Update /etc/sysconfig/network-scripts/ifcfg-ens8 as follows:

Make sure to configure DEFROUTE="no". For external access to work, the default route should use the public interface. 

DEFROUTE="no" # the public macvtap interface will provide the default route, not this one

Also make sure UUID is unique, you can generate a new UUID with uuidgen.

The configuration should be similar to: 

BOOTPROTO="none"
DEFROUTE=no # the public macvtap interface will provide the default route, not this one
DEVICE="ens8"
DNS1="8.8.8.8"
GATEWAY="192.168.1.1"
IPADDR="192.168.1.15"
IPV4_FAILURE_FATAL="yes"
IPV6_AUTOCONF="no"
IPV6_DEFROUTE="no"
IPV6_FAILURE_FATAL="no"
IPV6INIT="no"
IPV6_PEERDNS="no"
IPV6_PEERROUTES="no"
NAME="ens8"
NETMASK="255.255.255.0"
ONBOOT="yes"
PEERDNS="no"
PEERROUTES="no"
PREFIX="24"
TYPE="Ethernet"
UUID=adcf7be8-0dd7-46f5-b572-c6d273e23d87

For more details on how to configure network interfaces, see

Configuring a Network Interface

Second, public macvtap (ens9) interface Manual Configuration

A second macvtap interface connects the guest to public internet. The configuration should be similar to the example provided below, and written in /etc/sysconfig/network-scripts/ifcfg-ens9. "DEFROUTE=yes" is essential.

⚠️ Important: The routable IP address used below must not be used by another guests. 

BOOTPROTO="none"
DEFROUTE="yes" # essential
DEVICE="ens9"
DNS1="8.8.8.8"
GATEWAY="104.50.201.86"
IPADDR="104.50.201.85" # must not conflict with other guests' public address
IPV4_FAILURE_FATAL="yes"
IPV6_AUTOCONF="no"
IPV6_DEFROUTE="no"
IPV6_FAILURE_FATAL="no"
IPV6INIT="no"
IPV6_PEERDNS="no"
IPV6_PEERROUTES="no"
NAME="ens9"
NETMASK="255.255.255.248"
ONBOOT="yes"
PEERDNS="no"
PEERROUTES="no"
PREFIX="29"
TYPE="Ethernet"
UUID="30C65EA0-1EC3-46C6-8AC1-202D83102B35"

Reboot the second time, both interfaces should be operational.

Verification

After reboot, the guest must be accessible over ssh from an external network.

The routing table should be similar to: 

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         104.50.201.86   0.0.0.0         UG        0 0          0 ens9
104.50.201.80   0.0.0.0         255.255.255.248 U         0 0          0 ens9
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 ens8
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 ens9
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 ens8
192.168.122.0   0.0.0.0         255.255.255.0   U         0 0          0 eth0

Retrieved from "https://kb.novaordis.com/index.php?title=Attaching_a_Guest_Directly_to_a_Virtualization_Host_Network_Interface_with_a_macvtap_Driver&oldid=97556"