Boot taking a long time

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Boot taking a long time

Paul Groves
Hi everyone.

I am having some trouble with Ubuntu Server 18.04 networking.

The system has two onboard NICs used in a network bond (bond0) in
balance-rr mode. The network bond is working fine with both or either
NIC connected.

When I boot the system from off or reboot I am presented with:

A start job is running for Wait for Network to be Configured

This is displayed on the screen for approximately 3 minutes at which
point the system boots normally. How can I prevent this timer coming up?

I have this problem on two servers both running ubuntu 18.04.1 and with
the exact same configuration.

Here is my netplan configutration which was automatically generated at
setup.

Am I doing something wrong?

Paul

paul@web:~$ cat /etc/netplan/50-cloud-init.yaml
network:
     bonds:
         bond0:
             addresses:
             - 192.168.0.9/24
             dhcp4: false
             dhcp6: false
             gateway4: 192.168.0.1
             interfaces:
             - eno1
             - enp2s0
             nameservers:
                 addresses:
                 - 192.168.0.5
                 search:
                 - groves
             parameters:
                 mode: balance-rr
     ethernets:
         eno1:
             addresses: []
             dhcp4: false
             dhcp6: false
         enp2s0:
             addresses: []
             dhcp4: false
             dhcp6: false
     version: 2


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Boot taking a long time

Tom H-4
On Fri, Nov 2, 2018 at 4:52 PM Paul Groves <[hidden email]> wrote:

>
> I am having some trouble with Ubuntu Server 18.04 networking.
>
> The system has two onboard NICs used in a network bond (bond0) in
> balance-rr mode. The network bond is working fine with both or either
> NIC connected.
>
> When I boot the system from off or reboot I am presented with:
>
> A start job is running for Wait for Network to be Configured
>
> This is displayed on the screen for approximately 3 minutes at which
> point the system boots normally. How can I prevent this timer coming up?
>
> I have this problem on two servers both running ubuntu 18.04.1 and with
> the exact same configuration.

What's the unit that's blocking?

Check with "systemd-analyze blame" and "systemd-analyze critical-chain
[<unit>]" where/why the delay is.

You might also want to add "systemd.log_level=debug" to your kernel
cmdline in irder to get more logs.

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Boot taking a long time

Paul Groves


On 03/11/2018 10:26, Tom H wrote:
> What's the unit that's blocking?
>
> Check with "systemd-analyze blame" and "systemd-analyze critical-chain
> [<unit>]" where/why the delay is.
>
> You might also want to add "systemd.log_level=debug" to your kernel
> cmdline in irder to get more logs.


The last boot for some reason was quicker at around 1 and a half mins.
It can be anywhere from about this to 3 and a half mins.

See below the output of those two commands. It seems that
systemd-networkd-wait-online.service is the problem.

I will have to arrange downtime to reboot the system and change the
kernel parameters. Although the problem is easily replicable. All that
needs to be done is install Ubuntu 18.04.1 Server from the normal ISO
and configure a network bond. Then this happens.

It doesn't happen when you just have a normally configured interface
though, so I am wondering if it is where the interfaces eno1 and enp2s0
are disabled in netplan due to being slaves to bond0 the system is
waiting for them to come up, but of course they never do.

Is there a way in netplan of saying that an interface is optional?

Paul


paul@web:~$ systemd-analyze blame
      1min 4.712s systemd-networkd-wait-online.service

paul@web:~$ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@"
character.
The time the unit takes to start is printed after the "+" character.

graphical.target @1min 59.079s
└─multi-user.target @1min 59.079s
   └─apache2.service @1min 35.369s +23.709s
     └─basic.target @1min 34.917s
       └─sockets.target @1min 34.915s
         └─lxd.socket @1min 34.890s +23ms
           └─sysinit.target @1min 34.735s
             └─cloud-init.service @1min 33.499s +1.182s
               └─systemd-networkd-wait-online.service @28.784s +1min 4.712s
                 └─systemd-networkd.service @28.703s +74ms
                   └─network-pre.target @28.701s
                     └─cloud-init-local.service @5.109s +23.590s
                       └─systemd-remount-fs.service @4.509s +594ms
                         └─systemd-journald.socket @4.374s
                           └─system.slice @4.362s
                             └─-.slice @4.349s

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Boot taking a long time

Tom H-4
On Sat, Nov 3, 2018 at 4:17 PM Paul Groves <[hidden email]> wrote:
> On 03/11/2018 10:26, Tom H wrote:


>> What's the unit that's blocking?
>>
>> Check with "systemd-analyze blame" and "systemd-analyze critical-chain
>> [<unit>]" where/why the delay is.
>>
>> You might also want to add "systemd.log_level=debug" to your kernel
>> cmdline in irder to get more logs.
>
> The last boot for some reason was quicker at around 1 and a half mins.
> It can be anywhere from about this to 3 and a half mins.
>
> See below the output of those two commands. It seems that
> systemd-networkd-wait-online.service is the problem.

Indeed.


> I will have to arrange downtime to reboot the system and change the
> kernel parameters. Although the problem is easily replicable. All that
> needs to be done is install Ubuntu 18.04.1 Server from the normal ISO
> and configure a network bond. Then this happens.
>
> It doesn't happen when you just have a normally configured interface
> though, so I am wondering if it is where the interfaces eno1 and enp2s0
> are disabled in netplan due to being slaves to bond0 the system is
> waiting for them to come up, but of course they never do.
>
> Is there a way in netplan of saying that an interface is optional?
>
> paul@web:~$ systemd-analyze blame
>       1min 4.712s systemd-networkd-wait-online.service
>
> paul@web:~$ systemd-analyze critical-chain
> The time after the unit is active or started is printed after the "@"
> character.
> The time the unit takes to start is printed after the "+" character.
>
> graphical.target @1min 59.079s
> └─multi-user.target @1min 59.079s
>    └─apache2.service @1min 35.369s +23.709s
>      └─basic.target @1min 34.917s
>        └─sockets.target @1min 34.915s
>          └─lxd.socket @1min 34.890s +23ms
>            └─sysinit.target @1min 34.735s
>              └─cloud-init.service @1min 33.499s +1.182s
>                └─systemd-networkd-wait-online.service @28.784s +1min 4.712s
>                  └─systemd-networkd.service @28.703s +74ms
>                    └─network-pre.target @28.701s
>                      └─cloud-init-local.service @5.109s +23.590s
>                        └─systemd-remount-fs.service @4.509s +594ms
>                          └─systemd-journald.socket @4.374s
>                            └─system.slice @4.362s
>                              └─-.slice @4.349s


I set up a similar netplan config.


/etc/netplan # vi ubu.yaml


/etc/netplan # cat ubu.yaml
network:
  bonds:
    bond0:
      interfaces:
        - dummy0
        - dummy1
      addresses:
        - 192.168.0.9/24
      gateway4: 192.168.0.1
      dhcp4: no
      dhcp6: no
      parameters:
        mode: active-rr
  ethernets:
    dummy0:
      dhcp4: no
    dummy1:
      dhcp4: no


/etc/netplan # netplan generate


/etc/netplan # cd /run/systemd/network


/run/systemd/network # ls
10-netplan-bond0.netdev  10-netplan-bond0.network
10-netplan-dummy0.network  10-netplan-dummy1.network


/run/systemd/network # cat *
[NetDev]
Name=bond0
Kind=bond

[Bond]
Mode=active-rr
[Match]
Name=bond0

[Network]
LinkLocalAddressing=ipv6
Address=192.168.0.9/24
Address=192.168.0.9/24
Gateway=192.168.0.1
ConfigureWithoutCarrier=yes
[Match]
Name=dummy0

[Network]
LinkLocalAddressing=no
Bond=bond0
[Match]
Name=dummy1

[Network]
LinkLocalAddressing=no
Bond=bond0


/run/systemd/network #


bond0.network somehow ends up with two "Address=" address lines. I
haven't seen this before but this might be an 18.10 or 19.04 problem
rather than an 18.04 one. But I don't see why that would slow down the
boot process.

You could try copying the .network and .netdev files to
"/etc/systemd/network/" (and edit them, perhaps by removing the
LinkLocalAddressing and ConfigureWithoutCarrier to simplify
everything; even though they don't look like they'd slow anything
down) and move the yaml file out of "/etc/netplan/" to see what
happens.

Anything in the output of "journalctl"? Or 'systemctl status
systemd-networkd-wait-online.service"?

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: Boot taking a long time

Paul Groves
It turns out my suspicion was correct.

Here is my netplan configuration as generated by the Ubuntu Server Setup
wizard on the install disc:

network:
     bonds:
         bond0:
             addresses:
             - 192.168.0.9/24
             dhcp4: false
             dhcp6: false
             gateway4: 192.168.0.1
             interfaces:
             - eno1
             - enp2s0
             nameservers:
                 addresses:
                 - 192.168.0.5
                 search:
                 - groves
             parameters:
                 mode: balance-rr
     ethernets:
         eno1:
             addresses: []
             dhcp4: false
             dhcp6: false
         enp2s0:
             addresses: []
             dhcp4: false
             dhcp6: false


The problem is that the system was waiting for the network interfaces
eno1 and enps2s0 to come up and causing the delay at boot. Of course,
the interfaces configured this way will never come 'up'.

As suggested on another mailing list I modified the configuration to the
following.

     ethernets:
         eno1:
             addresses: []
             dhcp4: false
             dhcp6: false
             optional: true
         enp2s0:
             addresses: []
             dhcp4: false
             dhcp6: false
             optional: true


This has now resolved the problem. Because the interfaces are now set as
optional they are not required to be up in order to boot the system.

Then once the system is booted and the network bonding module is loaded
bond0 comes up.

Should this not be set automatically?

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users