This should hopefully avoid failing to build the initramfs
when important binaries like /bin/busybox get moved to /usr.
Also complain loudly when there's a path mismatch, so people notice
it.
Initially I thought of breaking off the Stat + error check call into its
own function as to reduce repetition, but given that it's only useful in
this situation where it only happens twice anyway, I'm not sure it
actually would reduce complexity.
Additionally, this means that .zst-compressed variants of files will be
searched for in all contexts where this function is used. I'm not sure
this is desirable.
Tested and works with arrow-db820c. I didn't test it on actual hardware,
but I verified that the firmware ended up in the initramfs via
$ pmbootstrap initfs ls. I choose this device because it uses firmware
from linux-firmware and also needs said firmware present in the
initramfs.
Closes https://gitlab.com/postmarketOS/postmarketos-mkinitfs/-/issues/39
Some kernel packages (e.g. linux-lts in Alpine) don't ship linux.efi, so
this needs to fallback to "vmlinuz" or else it won't be able to find a
kernel for boot-deploy.
There are several valid extensions that kernel modules can have, and the
list I had here was not complete... this meant that mkinitfs would fail
to include modules with extensions like ".ko.gz" when searching
directories.
This makes the check for a "valid" module file name a lot simpler,
allowing any file with ".ko" in the file name. While it's possible for a
non-module file to have ".ko" somewhere in the file name, it seems
unlikely if it's in the kernel modules directory... and this is an OK
compromise for now.
As we move towards UEFI on more devices, we want to use systemd-boot and
kernel images built with CONFIG_ZBOOT. However, these images aren't
compatible with existing Android bootloaders. As a result, we must
install both the old vmlinuz image for old bootloaders, and the fancy
new self-extracting EFI image.
When using systemd_boot, use linux.efi as the kernel file name instead
of globbing on vmlinuz*.
Signed-off-by: Caleb Connolly <caleb.connolly@linaro.org>
This removes code specific to installing osk-sdl in the initramfs
archive. osk-sdl has been deprecated / removed from postmarketOS for FDE
unlocking (in favor of using the app "unl0kr"), and other distros should
consider doing the same.
I think this was still causing some confusion, since it *might* look
like a failure when in reality it's not. I think it's important that
mkinitfs prints when it is adding something, and doesn't print when it
is *not* adding something, so that it should be clear if something
expected is missing and when something unexpected is included... without
having to sort out which is which every time the output is read.
Now:
10:57:41.737665 initramfs-extra completed in: 0.33s
...
10:57:42.008587 boot-deploy completed in: 0.27s
10:57:42.012973 mkinitfs completed in: 0.90s
Times is just truncated, not rounding, since it's simpler (no dependency
on the math module), and I'm not sure if anyone cares for what this
function prints. If there is a desire to return to higher precision
later, it could be enabled by a new flag.
fixes https://gitlab.com/postmarketOS/postmarketos-mkinitfs/-/issues/25
Another one found by fieldalignment:
modules.go:18:14: struct with 32 pointer bytes could be 24
Probably not going to matter much... but let's just get rid of the
warning.
Found by fieldalignment:
archive.go:46:14: struct with 88 pointer bytes could be 56
archive.go:66:18: struct with 24 pointer bytes could be 16
The first one probably doesn't matter that much, there's only like 2 of
those objects that are instantiated at runtime. However, there are many
ArchiveItems (hundreds or more depending on the archives compositions)
When this is printed, it's about to search the given path for stuff to
slurp up, but it hasn't actually included anything yet. Using
"Including" here was kinda confusing, so this changes it to use
"Searching for." Hopefully that's better!
By doing so, it adds more useful context to this:
- Including kernel modules
-- Unable to find dir, skipping...
- Including kernel modules
-- Unable to find dir, skipping...
I went with a simpler implementation that uses Go compression packages
to do the work. The downside of this is that the compression Level is a
bit weird to set, since most libraries discourage setting the numeric
compression level directly.
This is configured by setting `deviceinfo_initfs_compression`, the value
it expects is a string in the form: `FORMAT[:LEVEL]`, where `[:LEVEL]`
is optional. Actually setting the variable at all is optional... if
nothing is specified, or it can't parse the format/level from the string
value, it defaults to using gzip with the "default" level for the
package (which tries to mirror gzip's default, or something).
The level can be one of `default`, `fast`, `best`.
To configure gzip with the fastest compression (so, bigger size): deviceinfo_initfs_compression="gzip:fast"`
To configure zstd with the most compression: `deviceinfo_initfs_compression="zstd:best"`
To configure zstd with default compression: `deviceinfo_initfs_compression="zstd"` (or `deviceinfo_initfs_compression="zstd:default"`)
In this case, `gzip:default` is assumed: deviceinfo_initfs_compression="bananas:mmmm"`
There's really not a great way to map individual levels to each
compression library, so this just adds a new type that will invoke the
three relevant levels for each library used. This could be improved in
the future.
This replaces the parallel gzip with the boring gzip from Go's standard
lib. The main motivations for doing this are:
1) Reduce runtime memory requirements
2) shed some external dependencies
There's obviously a trade-off with compression speed/time (as seen
below), but I feel like it's a worthwhile trade-off.
Note that there's likely very little impact to boot performance wrt
extracting these archives, the compression levels are similar.
Measured on a Shift 6mq, which is a very fast phone...
** compress/gzip:
User time (seconds): 1.81
System time (seconds): 0.38
Percent of CPU this job got: 104%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.09
Maximum resident set size (kbytes): 62024
-rw-r--r-- 1 clayton clayton 6.1M Sep 20 17:20 initramfs
-rw-r--r-- 1 clayton clayton 2.5M Sep 20 17:20 initramfs-extra
** pgzip:
User time (seconds): 1.19
System time (seconds): 0.48
Percent of CPU this job got: 159%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.05
Maximum resident set size (kbytes): 109952
-rw-r--r-- 1 clayton clayton 6.8M Sep 20 17:20 initramfs
-rw-r--r-- 1 clayton clayton 2.8M Sep 20 17:20 initramfs-extra
inspired by: https://gitlab.com/postmarketOS/pmaports/-/issues/1704
Checking for the app is unnecessary, since the cmd.Run later will fail
if it doesn't exist. This allows dropping the dependency on misc.Exists.
There's also no reason to print that the command failed, just return the
error.
