systemd-networkd-wait-online service timing out during boot

Question

I have been this issue for quite a while so today I have tried a suggestion I found online and running systemd-analyze plot and checking the output.

I'm attaching the SVG file here because unless I am mistaken there are only two things taking an absurdly long time to boot, one of them systemd-networkd-wait-online (of course) but then afterwards, strangely, sendmail which I do not even use on this server it is within my home network and only functions as a seedbox/media server.

Am I missing something here? I disabled wlan previously to rule it out as it's a wired machine and has no need for wlan.

https://c0ld.net/slowboot.svg

I disabled sendmail, rebooted, watched it timeout and then ran it again and now the only thing taking time is the thing I'm trying to fix. Any help would be graciously appreciated.

https://c0ld.net/slowboot-nosendmail.svg

Here is a full journalctl: http://0x0.st/Xo36.txt

It has a static IP, it is not internet facing, runs plex, bittorrent-nox, jellyfin and has a couple of USB hard drives mounted at boot for storage and that's about all that's running other than webmin and cockpit. It has the full Ubuntu GNome installed but not booting at startup, I just wanted it for kicks and giggles if I need to ever use it as a backup PC.

I really do not want to simply mask the problem by disabling the service. I even tried changing to renderer: networkd from renderer: NetworkManager in netplan

Meant to add, output of networkctl

  1 lo     loopback carrier     unmanaged
  2 eno1   ether    routable    configured
  3 wlp1s0 wlan     off         unmanaged
3 links listed.

So nothing crazy there.

I also have optional: true under config of eno1 in netplan.

EDIT: Also tried adding --any and --interface=eno1 as specified below and both still have the same timeout. https://ubuntuforums.org/showthread.php?t=2490962

Apr 21 15:15:16 jays-lenovo systemd[1]: Starting Wait for Network to be Configured...
░░ Subject: A start job for unit systemd-networkd-wait-online.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit systemd-networkd-wait-online.service has begun execution.
░░
░░ The job identifier is 3886.
Apr 21 15:17:16 jays-lenovo systemd-networkd-wait-online[4967]: Timeout occurred while waiting for network connectivity.
Apr 21 15:17:16 jays-lenovo systemd[1]: systemd-networkd-wait-online.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ An ExecStart= process belonging to unit systemd-networkd-wait-online.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Apr 21 15:17:16 jays-lenovo systemd[1]: systemd-networkd-wait-online.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit systemd-networkd-wait-online.service has entered the 'failed' state with result 'exit-code'.
Apr 21 15:17:16 jays-lenovo systemd[1]: Failed to start Wait for Network to be Configured.
░░ Subject: A start job for unit systemd-networkd-wait-online.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit systemd-networkd-wait-online.service has finished with a failure.
░░
░░ The job identifier is 3886 and the job result is failed.

This was after restarting the service with --interface=eno1 which is most definitely up!!!!

The same check for systemd-networkd.service

-- Boot 984031206640442cb02a40798ddf3b7f --
Apr 21 14:21:12 jays-lenovo systemd[1]: Starting Network Configuration...
░░ Subject: A start job for unit systemd-networkd.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit systemd-networkd.service has begun execution.
░░
░░ The job identifier is 49.
Apr 21 14:21:12 jays-lenovo systemd-networkd[1178]: lo: Link UP
Apr 21 14:21:12 jays-lenovo systemd-networkd[1178]: lo: Gained carrier
Apr 21 14:21:12 jays-lenovo systemd-networkd[1178]: Enumeration completed
Apr 21 14:21:12 jays-lenovo systemd[1]: Started Network Configuration.
░░ Subject: A start job for unit systemd-networkd.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit systemd-networkd.service has finished successfully.
░░
░░ The job identifier is 49.
Apr 21 14:23:15 jays-lenovo systemd-networkd[1178]: eno1: Link UP
Apr 21 14:23:18 jays-lenovo systemd-networkd[1178]: eno1: Gained carrier
Apr 21 14:23:19 jays-lenovo systemd-networkd[1178]: eno1: Gained IPv6LL
Apr 21 14:51:50 jays-lenovo systemd-networkd[1178]: eno1: Re-configuring with /run/systemd/network/10-netplan-eno1.network
Apr 21 14:51:51 jays-lenovo systemd-networkd[1178]: eno1: Re-configuring with /run/systemd/network/10-netplan-eno1.network
Apr 21 14:51:51 jays-lenovo systemd-networkd[1178]: eno1: DHCPv6 lease lost
Apr 21 14:52:23 jays-lenovo systemd[1]: Stopping Network Configuration...
░░ Subject: A stop job for unit systemd-networkd.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A stop job for unit systemd-networkd.service has begun execution.
░░
░░ The job identifier is 4966.
Apr 21 14:52:23 jays-lenovo systemd-networkd[1178]: eno1: DHCPv6 lease lost
Apr 21 14:52:23 jays-lenovo systemd[1]: systemd-networkd.service: Deactivated successfully.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit systemd-networkd.service has successfully entered the 'dead' state.
Apr 21 14:52:23 jays-lenovo systemd[1]: Stopped Network Configuration.
░░ Subject: A stop job for unit systemd-networkd.service has finished
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A stop job for unit systemd-networkd.service has finished.
░░
░░ The job identifier is 4966 and the job result is done.
-- Boot dc2cb94cb1f649c49648ba8e085192db --
Apr 21 14:53:19 jays-lenovo systemd[1]: Starting Network Configuration...
░░ Subject: A start job for unit systemd-networkd.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit systemd-networkd.service has begun execution.
░░
░░ The job identifier is 25.
Apr 21 14:53:19 jays-lenovo systemd-networkd[1172]: lo: Link UP
Apr 21 14:53:19 jays-lenovo systemd-networkd[1172]: lo: Gained carrier
Apr 21 14:53:19 jays-lenovo systemd-networkd[1172]: Enumeration completed
Apr 21 14:53:19 jays-lenovo systemd[1]: Started Network Configuration.
░░ Subject: A start job for unit systemd-networkd.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit systemd-networkd.service has finished successfully.
░░
░░ The job identifier is 25.
Apr 21 14:53:19 jays-lenovo systemd-networkd[1172]: eno1: Link UP
Apr 21 14:53:22 jays-lenovo systemd-networkd[1172]: eno1: Gained carrier
Apr 21 14:53:24 jays-lenovo systemd-networkd[1172]: eno1: Gained IPv6LL
Apr 21 15:26:32 jays-lenovo systemd-networkd[1172]: eno1: Re-configuring with /run/systemd/network/10-netplan-eno1.network
Apr 21 15:26:32 jays-lenovo systemd-networkd[1172]: eno1: DHCPv6 lease lost
Apr 21 15:26:32 jays-lenovo systemd-networkd[1172]: eno1: Re-configuring with /run/systemd/network/10-netplan-eno1.network
Apr 21 15:26:32 jays-lenovo systemd-networkd[1172]: eno1: DHCPv6 lease lost

Full https://c0ld.net/dmesg.html available here. I do not know what else to try at this point, I have scoured the internet.

eno1 is configured as a static IP too so it makes no sense it's taking longer to be available than shorter.

192.168.2.1 is the main router for my entire household with multiple different routers and tens of devices, not a single other one has any issue.

# This is the network config written by 'subiquity'
network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      optional: false
      addresses:
        - 192.168.2.99/24
      nameservers:
        addresses: [192.168.2.1]
      routes:
        - to: default
          via: 192.168.2.1

score 6 · Answer 1 · answered May 06 '24 at 12:52

This kind of error is also present on a new Ubuntu Server 24.04 LTS install, if you have an unconnected ethernet interface. It is pretty standard to ship server-level hardware with multiple ethernet ports on it. Here for example is the output of one of my 1U servers: .

networkctl
IDX LINK     TYPE     OPERATIONAL SETUP
  1 lo       loopback carrier     unmanaged
  2 enp4s0f0 ether    routable    configured
  3 enp4s0f1 ether    no-carrier  configuring
3 links listed.

As you can see networkd isn't smart enough to see that an interface with no carrier is unplugged and therefore cannot complete dhcp configuration, and will continue attempting to use dhcp to request an IP for it.

I leave it alone and tolerate the extra 2 minute boot delay since there is always the chance in the future the first ethernet port will have a hardware failure and I'll have to physically move the ethernet cable to the second port, if that happens then the second port will immediately come up with an IP address.

score 3 · Answer 2 · answered Jun 11 '24 at 00:38

Inadvertently winding up with both NetworkManager and systemd-networkd enabled on my system (which has been through many dist-upgrade cycles and a bunch of package install and removals for various things) was the cause of the hang on mine. The answer of how to check that and choose one or the other can be found in https://askubuntu.com/a/1501504/209431.

score 2 · Accepted Answer · edited Sep 20 '24 at 00:09

What fixed it for me was to run the command contained in the systemd-networkd-wait-online.service file and let it finish. Once it finished, it now always finishes instantly. The service was always timing out without letting it finish before.

sudo nano /etc/systemd/system/systemd-networkd-wait-online.service

I took the line for ExecStart and ran that in terminal on its own.

Mine looked like

ExecStart=/lib/systemd/systemd-networkd-wait-online

So I ran the following at the terminal

/lib/systemd/systemd-networkd-wait-online

And I let that complete one time instead of letting the service timeout and kill it. Now it works perfectly everytime I or the service run it.

score 2 · Answer 4 · edited Nov 14 '24 at 00:53

After reading all of the answers, I've started thinking and I think I've solved it.
I have 3 interfaces (one being hp ilo):

    IDX LINK        TYPE     OPERATIONAL SETUP      
  1 lo          loopback carrier     unmanaged
  2 eno1        ether    routable    configured 
  3 eno2        ether    no-carrier  configuring
  4 docker0     bridge   routable    unmanaged
 64 veth0653007 ether    degraded    unmanaged

I've created 2 network config files for both eno interfaces to setup priorities:

sudo nano /etc/systemd/network/20-wired-first.network
sudo nano /etc/systemd/network/30-wired-second.network

First config:

[Match]
Name=eno1
[Network]
DHCP=yes
[DHCP]
RouteMetric=10

Second config:

[Match]
Name=eno2
[Network]
DHCP=yes
[DHCP]
RouteMetric=20

Then I've updated the (god-forsaken) service:

sudo nano /etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service
[Service]
    Type=oneshot
    ExecStart=/lib/systemd/systemd-networkd-wait-online --interface=eno1
    RemainAfterExit=yes

Then I've restarted the services:

sudo systemctl restart systemd-networkd
sudo systemctl restart systemd-networkd-wait-online

Now, without the config files interfaces lacked the proper prioritization as ExecStart only tells systemd which interface to wait for and doesn't configure how they should work.

I saw 2 options here:

Keep the ExecStart parameter for boot and keep the .network files
Disable wait-online

score 2 · Answer 5 · answered Mar 26 '25 at 20:44

This is what I did to fix the issue

I changed this file /lib/systemd/system/systemd-networkd-wait-online.service

To have this

ExecStart=/usr/lib/systemd/systemd-networkd-wait-online -i eth0

Where eth0 is the interface you care about, then I made the service not changable by updates

sudo chattr +i /lib/systemd/system/systemd-networkd-wait-online.service

B3 IT Systeme GmbH · Answer 6 · 2025-03-24T12:33:54.940

We had the same problems with systemd-networkd-wait-online on Ubuntu 22.04 after upgrading from Ubuntu 20.04.

networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  2 eth0 ether routable configuring

Always remained in configuring and this caused the timeout for systemd-networkd-wait-online.

We solved the problem by customizing /etc/netplan/01-netcfg.yaml. We removed gateway6, which is deprecated, and explicitly added a route for IPv6:

routes:
        - to: 0.0.0.0/0
          via: 10.0.0.1
          on-link: true
        - to: "::/0"
          via: fe80::1
          on-link: true

After that:

netplan apply
systemctl restart systemd-networkd

After that it looked correct

networkctl
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  2 eth0 ether routable configured

and systemd-networkd-wait-online also worked.

score 1 · Answer 7 · answered Jun 20 '25 at 17:13

1

In the netplan yaml file, change optional: false to optional: true and the test should be skipped if the interface has no link.

answered Jun 20 '25 at 17:13

guitarpicva

11

systemd-networkd-wait-online service timing out during boot

7 Answers7

Linked