Supporting LZ4 as initramfs compressor

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Supporting LZ4 as initramfs compressor

Balint Reczey
Hi,

Initramfs-tools uses gzip compression by default which served us well
for quite some time but LZ4 offers way faster decompression while
making a only slightly bigger initramfs files.

On my old laptop the initramfs extraction time decreased from ~1.2s to ~0.24s:
(with lz4)
kernel: [    0.297726] Unpacking initramfs...
kernel: [    0.535061] Freeing initrd memory: 77940K
kernel: [    0.301637] Unpacking initramfs...
kernel: [    0.539109] Freeing initrd memory: 77940K
(with gzip)
kernel: [    0.273748] Unpacking initramfs...
kernel: [    1.490066] Freeing initrd memory: 57140K
kernel: [    0.281729] Unpacking initramfs...
kernel: [    1.498493] Freeing initrd memory: 57140K

The increase in the initrd.img size is ~14%:
(lz4)
-rw-r--r-- 1 root root 66709065 márc  19 14:24
/boot/initrd.img-4.15.0-12-generic
(gzip)
-rw-r--r-- 1 root root 58510993 márc  19 12:57
/boot/initrd.img-4.15.0-12-generic.bak

Initramfs creation speed also improved a bit from ~24s to ~21s wall clock time:
(lz4)
update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
14.97user 6.31system 0:20.47elapsed 103%CPU (0avgtext+0avgdata
22368maxresident)k
update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
15.18user 6.49system 0:20.48elapsed 105%CPU (0avgtext+0avgdata
22308maxresident)k
(gzip)
update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
18.23user 6.77system 0:23.61elapsed 105%CPU (0avgtext+0avgdata
22396maxresident)k
update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
18.38user 6.83system 0:23.82elapsed 105%CPU (0avgtext+0avgdata
22292maxresident)k

Base on the results I plan adding LZ4 compression support to
initramfs-tools as requested in LP: #1488620 [1] in the next days
without setting it as default and I propose setting LZ4 as default for
18.10.

I'm aware of the old problem of /boot filling up with kernels and
initramfs files but unattended-upgrades already removes old kernels
[2] and update-manager is planned to do the same starting with 18.04
[3] thus Ubuntu systems will have enough space in /boot to allow
slightly bigger initrd files.

Cheers,
Balint

[1] https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1488620
[2] https://launchpad.net/ubuntu/+source/unattended-upgrades/1.0ubuntu1
[3] https://code.launchpad.net/~rbalint/update-manager/remove-autoremovable-kernels/+merge/341599

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Supporting LZ4 as initramfs compressor

Julian Andres Klode
On Mon, Mar 19, 2018 at 02:59:24PM +0000, Balint Reczey wrote:

> Hi,
>
> Initramfs-tools uses gzip compression by default which served us well
> for quite some time but LZ4 offers way faster decompression while
> making a only slightly bigger initramfs files.
>
> On my old laptop the initramfs extraction time decreased from ~1.2s to ~0.24s:
> (with lz4)
> kernel: [    0.297726] Unpacking initramfs...
> kernel: [    0.535061] Freeing initrd memory: 77940K
> kernel: [    0.301637] Unpacking initramfs...
> kernel: [    0.539109] Freeing initrd memory: 77940K
> (with gzip)
> kernel: [    0.273748] Unpacking initramfs...
> kernel: [    1.490066] Freeing initrd memory: 57140K
> kernel: [    0.281729] Unpacking initramfs...
> kernel: [    1.498493] Freeing initrd memory: 57140K
>
> The increase in the initrd.img size is ~14%:
> (lz4)
> -rw-r--r-- 1 root root 66709065 márc  19 14:24
> /boot/initrd.img-4.15.0-12-generic
> (gzip)
> -rw-r--r-- 1 root root 58510993 márc  19 12:57
> /boot/initrd.img-4.15.0-12-generic.bak
>
> Initramfs creation speed also improved a bit from ~24s to ~21s wall clock time:
> (lz4)
> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
> 14.97user 6.31system 0:20.47elapsed 103%CPU (0avgtext+0avgdata
> 22368maxresident)k
> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
> 15.18user 6.49system 0:20.48elapsed 105%CPU (0avgtext+0avgdata
> 22308maxresident)k
> (gzip)
> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
> 18.23user 6.77system 0:23.61elapsed 105%CPU (0avgtext+0avgdata
> 22396maxresident)k
> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
> 18.38user 6.83system 0:23.82elapsed 105%CPU (0avgtext+0avgdata
> 22292maxresident)k

For me it was 16 -> 10 I think.

>
> Base on the results I plan adding LZ4 compression support to
> initramfs-tools as requested in LP: #1488620 [1] in the next days
> without setting it as default
>

+1

> and I propose setting LZ4 as default for 18.10.

We might have zstd support by that time (I hope), it might make sense
to use that then (better space/time tradeoff), but we'll have to see.

--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Supporting LZ4 as initramfs compressor

Balint Reczey
On Mon, Mar 19, 2018 at 3:12 PM, Julian Andres Klode
<[hidden email]> wrote:

> On Mon, Mar 19, 2018 at 02:59:24PM +0000, Balint Reczey wrote:
>> Hi,
>>
>> Initramfs-tools uses gzip compression by default which served us well
>> for quite some time but LZ4 offers way faster decompression while
>> making a only slightly bigger initramfs files.
>>
>> On my old laptop the initramfs extraction time decreased from ~1.2s to ~0.24s:
>> (with lz4)
>> kernel: [    0.297726] Unpacking initramfs...
>> kernel: [    0.535061] Freeing initrd memory: 77940K
>> kernel: [    0.301637] Unpacking initramfs...
>> kernel: [    0.539109] Freeing initrd memory: 77940K
>> (with gzip)
>> kernel: [    0.273748] Unpacking initramfs...
>> kernel: [    1.490066] Freeing initrd memory: 57140K
>> kernel: [    0.281729] Unpacking initramfs...
>> kernel: [    1.498493] Freeing initrd memory: 57140K
>>
>> The increase in the initrd.img size is ~14%:
>> (lz4)
>> -rw-r--r-- 1 root root 66709065 márc  19 14:24
>> /boot/initrd.img-4.15.0-12-generic
>> (gzip)
>> -rw-r--r-- 1 root root 58510993 márc  19 12:57
>> /boot/initrd.img-4.15.0-12-generic.bak
>>
>> Initramfs creation speed also improved a bit from ~24s to ~21s wall clock time:
>> (lz4)
>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>> 14.97user 6.31system 0:20.47elapsed 103%CPU (0avgtext+0avgdata
>> 22368maxresident)k
>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>> 15.18user 6.49system 0:20.48elapsed 105%CPU (0avgtext+0avgdata
>> 22308maxresident)k
>> (gzip)
>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>> 18.23user 6.77system 0:23.61elapsed 105%CPU (0avgtext+0avgdata
>> 22396maxresident)k
>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>> 18.38user 6.83system 0:23.82elapsed 105%CPU (0avgtext+0avgdata
>> 22292maxresident)k
>
> For me it was 16 -> 10 I think.
>
>>
>> Base on the results I plan adding LZ4 compression support to
>> initramfs-tools as requested in LP: #1488620 [1] in the next days
>> without setting it as default
>>
>
> +1
>
>> and I propose setting LZ4 as default for 18.10.
>
> We might have zstd support by that time (I hope), it might make sense
> to use that then (better space/time tradeoff), but we'll have to see.

That would also be an option, but I expect zstd to bring little speed
advantage over LZ4 here while LZ4 support could be easily backported
to releases with older kernels.

The proposed patch enabling lz4 uses lz4 -9. I tried lz4's default
compression, -1, but decompression speed difference was barely
noticeable - most likely due to copying being the bottleneck.

In fact I just realized that i copied the results created with lz4 -1,
where the compressed initrd size was 77940K.
The correct results for lz4 -9 are the following, taking the same
~0.24s to decompress:

kernel: [    0.285692] Unpacking initramfs...
kernel: [    0.518806] Freeing initrd memory: 65148K
kernel: [    0.289731] Unpacking initramfs...
kernel: [    0.522823] Freeing initrd memory: 65148K

Cheers,
Balint

--
Balint Reczey
Ubuntu & Debian Developer

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Supporting LZ4 as initramfs compressor

Marcos Alano
Last night I was trying to decrypt my laptop remotely. Work well
(after some research) and so on, but the point is I put Dropbear SSH
server inside initramfs and a faster decompress would be very
welcomed.

On Mon, Mar 19, 2018 at 2:19 PM, Balint Reczey
<[hidden email]> wrote:

> On Mon, Mar 19, 2018 at 3:12 PM, Julian Andres Klode
> <[hidden email]> wrote:
>> On Mon, Mar 19, 2018 at 02:59:24PM +0000, Balint Reczey wrote:
>>> Hi,
>>>
>>> Initramfs-tools uses gzip compression by default which served us well
>>> for quite some time but LZ4 offers way faster decompression while
>>> making a only slightly bigger initramfs files.
>>>
>>> On my old laptop the initramfs extraction time decreased from ~1.2s to ~0.24s:
>>> (with lz4)
>>> kernel: [    0.297726] Unpacking initramfs...
>>> kernel: [    0.535061] Freeing initrd memory: 77940K
>>> kernel: [    0.301637] Unpacking initramfs...
>>> kernel: [    0.539109] Freeing initrd memory: 77940K
>>> (with gzip)
>>> kernel: [    0.273748] Unpacking initramfs...
>>> kernel: [    1.490066] Freeing initrd memory: 57140K
>>> kernel: [    0.281729] Unpacking initramfs...
>>> kernel: [    1.498493] Freeing initrd memory: 57140K
>>>
>>> The increase in the initrd.img size is ~14%:
>>> (lz4)
>>> -rw-r--r-- 1 root root 66709065 márc  19 14:24
>>> /boot/initrd.img-4.15.0-12-generic
>>> (gzip)
>>> -rw-r--r-- 1 root root 58510993 márc  19 12:57
>>> /boot/initrd.img-4.15.0-12-generic.bak
>>>
>>> Initramfs creation speed also improved a bit from ~24s to ~21s wall clock time:
>>> (lz4)
>>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>>> 14.97user 6.31system 0:20.47elapsed 103%CPU (0avgtext+0avgdata
>>> 22368maxresident)k
>>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>>> 15.18user 6.49system 0:20.48elapsed 105%CPU (0avgtext+0avgdata
>>> 22308maxresident)k
>>> (gzip)
>>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>>> 18.23user 6.77system 0:23.61elapsed 105%CPU (0avgtext+0avgdata
>>> 22396maxresident)k
>>> update-initramfs: Generating /boot/initrd.img-4.15.0-12-generic
>>> 18.38user 6.83system 0:23.82elapsed 105%CPU (0avgtext+0avgdata
>>> 22292maxresident)k
>>
>> For me it was 16 -> 10 I think.
>>
>>>
>>> Base on the results I plan adding LZ4 compression support to
>>> initramfs-tools as requested in LP: #1488620 [1] in the next days
>>> without setting it as default
>>>
>>
>> +1
>>
>>> and I propose setting LZ4 as default for 18.10.
>>
>> We might have zstd support by that time (I hope), it might make sense
>> to use that then (better space/time tradeoff), but we'll have to see.
>
> That would also be an option, but I expect zstd to bring little speed
> advantage over LZ4 here while LZ4 support could be easily backported
> to releases with older kernels.
>
> The proposed patch enabling lz4 uses lz4 -9. I tried lz4's default
> compression, -1, but decompression speed difference was barely
> noticeable - most likely due to copying being the bottleneck.
>
> In fact I just realized that i copied the results created with lz4 -1,
> where the compressed initrd size was 77940K.
> The correct results for lz4 -9 are the following, taking the same
> ~0.24s to decompress:
>
> kernel: [    0.285692] Unpacking initramfs...
> kernel: [    0.518806] Freeing initrd memory: 65148K
> kernel: [    0.289731] Unpacking initramfs...
> kernel: [    0.522823] Freeing initrd memory: 65148K
>
> Cheers,
> Balint
>
> --
> Balint Reczey
> Ubuntu & Debian Developer
>
> --
> ubuntu-devel mailing list
> [hidden email]
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel



--
Marcos H. Alano
Linux System Administrator
[hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Reply | Threaded
Open this post in threaded view
|

Re: Supporting LZ4 as initramfs compressor

Steve Langasek-6
In reply to this post by Balint Reczey
Hi Balint,

On Mon, Mar 19, 2018 at 02:59:24PM +0000, Balint Reczey wrote:

> Initramfs-tools uses gzip compression by default which served us well
> for quite some time but LZ4 offers way faster decompression while
> making a only slightly bigger initramfs files.

When people have previously discussed lz4 on IRC as a possible choice for
default compression algorithm, I had the impression that this was with the
expectation that the resulting initramfs files would be *smaller* than with
using gzip.

(I think.  It happens that names for compression algorithms are remarkably
unoriginal, so it's possible I've confused lz4 with another.)

But your data shows that lz4-compressed initramfs is definitely larger than
gzip, which means that there are tradeoffs here that should be fully
examined.

After all, an initramfs that's not compressed at all would take even less
time to decompress at boot (0s) but would occupy more space.  But you aren't
advocating for this, so there must be some reason you believe lz4 is more of
a sweet spot than gzip?


The first thing that I see missing from this analysis is the time it takes
for the firmware/bootloader to load the initramfs into memory.  I know from
experience that some bootloader filesystem drivers have quite poor
performance; and the time spent loading the initramfs into memory will scale
roughly linearly with the file size.  So any analysis of lz4 impact on boot
speed needs to take this into account.  We should show that the overall
bootspeed from bootloader to pid 1 has actually improved, and this should be
measured with multiple bootloader driver implementations (across
architectures; UEFI vs BIOS; possibly on multiple virtualization substrates
vs. x86 bare metal).


The second thing to consider is that, regardless of any improvements in our
autoremoval of kernels, we have had some historical default sizing for /boot
partitions in the installer which are now on the small side for even a fully
correct kernel upgrade path.

In trusty, the default (max) size for a /boot partition was 256M.  In
xenial, it was 512M.  In artful, we have bumped this up to 768M with a
minimum of 512M because of LP: #1716999.

The actual space consumed by the static files in the 4.13.0-7-generic kernel
in artful - not counting the current .efi.signed duplication which will
hopefully go away soon - is just under 13MiB.  My bootloader here is 8MiB,
but 10MiB is not out of the question in some configurations.  My initramfs
is 52MiB rather than the 55MiB in that bug, but again 55MiB is plausible -
and your own numbers seem to show 56MiB.

That means that anyone who installed with trusty, has /boot as a separate
partition, and has plymouth in the initramfs (such as for encrypted root
disk, which would be a common reason to have a separate /boot) has already
run out of space on their /boot while using gzip; so must already reinstall
or switch to a non-default initrd compression algorithm on upgrade.  This
can therefore be ignored for the choice between gzip and lz4 as default.

Anyone who installed with xenial is borderline today with artful;
56*4+3*13+10 == 273MiB, which is more than xenial's minimum /boot size of
256MiB but less than the max of 512MiB.  So some number of users have a boot
partition that's large enough to accommodate gzipped initrd, but will run
out of space once you switch to lz4 (64*4+3*13+10 == 305MiB).  And those
that don't run out of space immediately as a result of the switch to lz4
would still run out of space sooner as the kernel size grows (since the
kernel definitely seems to only grow over time with new drivers).

Systems installed with artful or newer seem to still be fine for a while,
with either gzip or lz4.

So there's a decision to be made about whether we care about upgrades
working with the default compression on systems installed with smaller boot,
and for how long.  If we decide this shouldn't block switching the default
compression, we also need to sort out how we will communicate this to
affected users - and in particular, how to avoid problems on upgrade when
the user runs out of disk space at the worst possible time.


> Base on the results I plan adding LZ4 compression support to
> initramfs-tools as requested in LP: #1488620 [1] in the next days
> without setting it as default and I propose setting LZ4 as default for
> 18.10.

Since this is a non-default option and doesn't introduce any new
dependencies, this seems fine.  It also doesn't seem urgent to me; in terms
of the upgrade path, it doesn't need to be supported in 18.04 in order to
consider making it the default in 18.10.

Cheers,
--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
[hidden email]                                     [hidden email]

--
ubuntu-devel mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

signature.asc (817 bytes) Download Attachment