I am preparing migration of an UBUNTU 16.04.7 server with 5 NIC's to 20.04. My testing server has 4 NIC's: 2 onboard and 2 as Intel Pro NIC. Configured in 00-installer-config.yaml. The first NIC is configured as:
version: 2
renderer: networkd
ethernets:
enp65s0f0:
addresses: [10.1.11.230/24]
gateway4: 10.1.11.11
nameservers:
addresses: [10.1.11.11, 1.1.1.1]
enp193s0f0:
optional: true
enp193s0f1:
optional: true
enp65s0f1:
optional: true
This part works. The next 3 NIC's are going to be bond0, with vlans on top and bridges:
bonds:
bond0:
interfaces: [enp193s0f0, enp193s0f1, enp65s0f1]
parameters:
mode: 802.3ad
lacp-rate: fast
mii-monitor-interval: 100
min-links: 1
vlans:
vlan1:
id: 1
link: bond0
vlan11:
id: 11
link: bond0
vlan240:
id: 240
link: bond0
bridges:
br100:
interfaces: [vlan1]
br11:
interfaces: [vlan11]
br240:
interfaces: [vlan240]
The bond0 does not work. I have searched and tried all kinds of variations with ordering of NIC's, dhcp4: false, dhcp6: false, optional:true only using the IntelPro NIC's for the bonding. I have reviewed answers on this and other forums, but no solution so far yet.
- What's wrong with my configuration? Anything missing?
- Is the architecture I am aiming at still supported? I want my KVM guests to use a bridge so they can communicate with each other without needing the physical switch and without needing VLAN to be configured in the guests. This worked perfectly in Ub16.04.
My switch is a HP V1910-24G and has 4 other working trunks (LACP/802.3ad), with the Ub16.04 (4 NIC's)server and 3 QNAP NAS's with each a double NIC.
See the following outputs on my Ubuntu 20.04 with the above netplan config:
networkctl list
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp65s0f0 ether routable configured
3 enp65s0f1 ether enslaved configured
4 enp193s0f0 ether enslaved configured
5 enp193s0f1 ether enslaved configured
6 bond0 bond degraded configured
7 br100 bridge degraded configured
8 br11 bridge degraded configured
9 br240 bridge degraded configured
10 vlan.240 vlan carrier unmanaged
11 vlan.11 vlan carrier unmanaged
12 vlan.1 vlan carrier unmanaged
13 virbr0 bridge no-carrier unmanaged
14 virbr0-nic ether off unmanaged
17 vnet0 ether degraded unmanaged
ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp65s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether d0:50:99:d4:bc:e3 brd ff:ff:ff:ff:ff:ff
3: enp65s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
4: enp193s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
5: enp193s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
7: br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
8: br11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
9: br240: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
10: vlan.240@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br240 state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
11: vlan.11@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br11 state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
12: vlan.1@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br100 state UP mode DEFAULT group default qlen 1000
link/ether 16:3d:9c:75:6b:c5 brd ff:ff:ff:ff:ff:ff
13: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
link/ether 52:54:00:f4:49:5c brd ff:ff:ff:ff:ff:ff
14: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN mode DEFAULT group default qlen 1000
link/ether 52:54:00:f4:49:5c brd ff:ff:ff:ff:ff:ff
17: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br11 state UNKNOWN mode DEFAULT group default qlen 1000
link/ether fe:54:00:d2:56:7d brd ff:ff:ff:ff:ff:ff
netplan --debug apply
** (generate:11473): DEBUG: 13:54:31.627: Processing input file /etc/netplan/00-installer-config.yaml..
** (generate:11473): DEBUG: 13:54:31.628: starting new processing pass
** (generate:11473): DEBUG: 13:54:31.628: We have some netdefs, pass them through a final round of validation
** (generate:11473): DEBUG: 13:54:31.628: vlan240: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: bond0: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: br11: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: enp65s0f1: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: enp65s0f0: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: vlan11: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: enp193s0f1: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: br100: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: vlan1: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: enp193s0f0: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: br240: setting default backend to 1
** (generate:11473): DEBUG: 13:54:31.628: Configuration is valid
** (generate:11473): DEBUG: 13:54:31.628: Generating output files..
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition enp65s0f0 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition enp65s0f0 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition enp193s0f0 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition enp193s0f0 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition enp193s0f1 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition enp193s0f1 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition enp65s0f1 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition enp65s0f1 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition bond0 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition bond0 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition vlan1 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition vlan1 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition vlan11 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition vlan11 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition vlan240 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition vlan240 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition br100 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition br100 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition br11 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition br11 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: openvswitch: definition br240 is not for us (backend 1)
** (generate:11473): DEBUG: 13:54:31.629: NetworkManager: definition br240 is not for us (backend 1)
(generate:11473): GLib-DEBUG: 13:54:31.629: posix_spawn avoided (fd close requested)
(generate:11473): GLib-DEBUG: 13:54:31.630: posix_spawn avoided (fd close requested)
DEBUG:netplan generated networkd configuration changed, restarting networkd
DEBUG:enp65s0f0 not found in {}
DEBUG:enp193s0f0 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}}
DEBUG:enp193s0f1 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}, 'enp193s0f0': {'optional': True}}
DEBUG:enp65s0f1 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}, 'enp193s0f0': {'optional': True}, 'enp193s0f1': {'optional': True}}
DEBUG:br100 not found in {}
DEBUG:br11 not found in {'br100': {'interfaces': ['vlan1']}}
DEBUG:br240 not found in {'br100': {'interfaces': ['vlan1']}, 'br11': {'interfaces': ['vlan11']}}
DEBUG:bond0 not found in {}
DEBUG:vlan1 not found in {}
DEBUG:vlan11 not found in {'vlan1': {'id': 1, 'link': 'bond0'}}
DEBUG:vlan240 not found in {'vlan1': {'id': 1, 'link': 'bond0'}, 'vlan11': {'id': 11, 'link': 'bond0'}}
DEBUG:Merged config:
network:
bonds:
bond0:
interfaces:
- enp193s0f0
- enp193s0f1
- enp65s0f1
parameters:
lacp-rate: fast
mii-monitor-interval: 100
min-links: 1
mode: 802.3ad
bridges:
br100:
interfaces:
- vlan1
br11:
interfaces:
- vlan11
br240:
interfaces:
- vlan240
ethernets:
enp193s0f0:
optional: true
enp193s0f1:
optional: true
enp65s0f0:
addresses:
- 10.1.11.230/24
gateway4: 10.1.11.11
nameservers:
addresses:
- 10.1.11.11
- 1.1.1.1
enp65s0f1:
optional: true
ovs_ports: {}
vlans:
vlan1:
id: 1
link: bond0
vlan11:
id: 11
link: bond0
vlan240:
id: 240
link: bond0
wifis: {}
DEBUG:no netplan generated NM configuration exists
DEBUG:enp65s0f0 not found in {}
DEBUG:enp193s0f0 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}}
DEBUG:enp193s0f1 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}, 'enp193s0f0': {'optional': True}}
DEBUG:enp65s0f1 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}, 'enp193s0f0': {'optional': True}, 'enp193s0f1': {'optional': True}}
DEBUG:br100 not found in {}
DEBUG:br11 not found in {'br100': {'interfaces': ['vlan1']}}
DEBUG:br240 not found in {'br100': {'interfaces': ['vlan1']}, 'br11': {'interfaces': ['vlan11']}}
DEBUG:bond0 not found in {}
DEBUG:vlan1 not found in {}
DEBUG:vlan11 not found in {'vlan1': {'id': 1, 'link': 'bond0'}}
DEBUG:vlan240 not found in {'vlan1': {'id': 1, 'link': 'bond0'}, 'vlan11': {'id': 11, 'link': 'bond0'}}
DEBUG:Merged config:
network:
bonds:
bond0:
interfaces:
- enp193s0f0
- enp193s0f1
- enp65s0f1
parameters:
lacp-rate: fast
mii-monitor-interval: 100
min-links: 1
mode: 802.3ad
bridges:
br100:
interfaces:
- vlan1
br11:
interfaces:
- vlan11
br240:
interfaces:
- vlan240
ethernets:
enp193s0f0:
optional: true
enp193s0f1:
optional: true
enp65s0f0:
addresses:
- 10.1.11.230/24
gateway4: 10.1.11.11
nameservers:
addresses:
- 10.1.11.11
- 1.1.1.1
enp65s0f1:
optional: true
ovs_ports: {}
vlans:
vlan1:
id: 1
link: bond0
vlan11:
id: 11
link: bond0
vlan240:
id: 240
link: bond0
wifis: {}
DEBUG:Skipping non-physical interface: lo
DEBUG:device enp65s0f0 operstate is up, not changing
DEBUG:Skipping composite member enp65s0f1
DEBUG:Skipping composite member enp193s0f0
DEBUG:Skipping composite member enp193s0f1
DEBUG:Skipping non-physical interface: bond0
DEBUG:Skipping non-physical interface: br100
DEBUG:Skipping non-physical interface: br11
DEBUG:Skipping non-physical interface: br240
DEBUG:Skipping non-physical interface: vlan.240
DEBUG:Skipping non-physical interface: vlan.11
DEBUG:Skipping non-physical interface: vlan.1
DEBUG:Skipping non-physical interface: virbr0
DEBUG:Skipping non-physical interface: virbr0-nic
DEBUG:Skipping non-physical interface: vnet0
DEBUG:{}
DEBUG:netplan triggering .link rules for lo
DEBUG:netplan triggering .link rules for enp65s0f0
DEBUG:netplan triggering .link rules for enp65s0f1
DEBUG:netplan triggering .link rules for enp193s0f0
DEBUG:netplan triggering .link rules for enp193s0f1
DEBUG:netplan triggering .link rules for bond0
DEBUG:netplan triggering .link rules for br100
DEBUG:netplan triggering .link rules for br11
DEBUG:netplan triggering .link rules for br240
DEBUG:netplan triggering .link rules for vlan.240
DEBUG:netplan triggering .link rules for vlan.11
DEBUG:netplan triggering .link rules for vlan.1
DEBUG:netplan triggering .link rules for virbr0
DEBUG:netplan triggering .link rules for virbr0-nic
DEBUG:netplan triggering .link rules for vnet0
DEBUG:enp65s0f0 not found in {}
DEBUG:enp193s0f0 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}}
DEBUG:enp193s0f1 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}, 'enp193s0f0': {'optional': True}}
DEBUG:enp65s0f1 not found in {'enp65s0f0': {'addresses': ['10.1.11.230/24'], 'gateway4': '10.1.11.11', 'nameservers': {'addresses': ['10.1.11.11', '1.1.1.1']}}, 'enp193s0f0': {'optional': True}, 'enp193s0f1': {'optional': True}}
DEBUG:br100 not found in {}
DEBUG:br11 not found in {'br100': {'interfaces': ['vlan1']}}
DEBUG:br240 not found in {'br100': {'interfaces': ['vlan1']}, 'br11': {'interfaces': ['vlan11']}}
DEBUG:bond0 not found in {}
DEBUG:vlan1 not found in {}
DEBUG:vlan11 not found in {'vlan1': {'id': 1, 'link': 'bond0'}}
DEBUG:vlan240 not found in {'vlan1': {'id': 1, 'link': 'bond0'}, 'vlan11': {'id': 11, 'link': 'bond0'}}
DEBUG:Merged config:
network:
bonds:
bond0:
interfaces:
- enp193s0f0
- enp193s0f1
- enp65s0f1
parameters:
lacp-rate: fast
mii-monitor-interval: 100
min-links: 1
mode: 802.3ad
bridges:
br100:
interfaces:
- vlan1
br11:
interfaces:
- vlan11
br240:
interfaces:
- vlan240
ethernets:
enp193s0f0:
optional: true
enp193s0f1:
optional: true
enp65s0f0:
addresses:
- 10.1.11.230/24
gateway4: 10.1.11.11
nameservers:
addresses:
- 10.1.11.11
- 1.1.1.1
enp65s0f1:
optional: true
ovs_ports: {}
vlans:
vlan1:
id: 1
link: bond0
vlan11:
id: 11
link: bond0
vlan240:
id: 240
link: bond0
wifis: {}
I am at the end of my options and ready to remove cloud-init/netplan and revert to the working configuration I had in 16.04. Thanks for reading this far.
Frank
