There's some repetition that's added by unrolling this, but it will
allow passing the main initramfs archive's filelister to
archive.AddItemsExclude when generating the initramfs-extra.
I looked at several ways to implement this, this seems like least
terrible thing to do... The runner-up was to return a FileList from
archive.AddItems, and pass that around... Eww.
I think this was still causing some confusion, since it *might* look
like a failure when in reality it's not. I think it's important that
mkinitfs prints when it is adding something, and doesn't print when it
is *not* adding something, so that it should be clear if something
expected is missing and when something unexpected is included... without
having to sort out which is which every time the output is read.
Now:
10:57:41.737665 initramfs-extra completed in: 0.33s
...
10:57:42.008587 boot-deploy completed in: 0.27s
10:57:42.012973 mkinitfs completed in: 0.90s
Times is just truncated, not rounding, since it's simpler (no dependency
on the math module), and I'm not sure if anyone cares for what this
function prints. If there is a desire to return to higher precision
later, it could be enabled by a new flag.
fixes https://gitlab.com/postmarketOS/postmarketos-mkinitfs/-/issues/25
Another one found by fieldalignment:
modules.go:18:14: struct with 32 pointer bytes could be 24
Probably not going to matter much... but let's just get rid of the
warning.
Found by fieldalignment:
archive.go:46:14: struct with 88 pointer bytes could be 56
archive.go:66:18: struct with 24 pointer bytes could be 16
The first one probably doesn't matter that much, there's only like 2 of
those objects that are instantiated at runtime. However, there are many
ArchiveItems (hundreds or more depending on the archives compositions)
When this is printed, it's about to search the given path for stuff to
slurp up, but it hasn't actually included anything yet. Using
"Including" here was kinda confusing, so this changes it to use
"Searching for." Hopefully that's better!
By doing so, it adds more useful context to this:
- Including kernel modules
-- Unable to find dir, skipping...
- Including kernel modules
-- Unable to find dir, skipping...
This adds two new CI jobs that run when a new tag is pushed to this
repo: one to tar up the vendored dependencies and another one to push
the tarball as an asset in a new gitlab "release" for the tag.
fixes: #5
There is little(?) reason to compress the -extra archive, but some
there are some good reasons why it should be left uncompressed:
1) decompression increases boot time
2) selecting exotic formats (e.g. lzma or zstd) can actually
increase the initramfs size since you now have to include
support for those archive formats
In the future this could be made configurable...
I went with a simpler implementation that uses Go compression packages
to do the work. The downside of this is that the compression Level is a
bit weird to set, since most libraries discourage setting the numeric
compression level directly.
This is configured by setting `deviceinfo_initfs_compression`, the value
it expects is a string in the form: `FORMAT[:LEVEL]`, where `[:LEVEL]`
is optional. Actually setting the variable at all is optional... if
nothing is specified, or it can't parse the format/level from the string
value, it defaults to using gzip with the "default" level for the
package (which tries to mirror gzip's default, or something).
The level can be one of `default`, `fast`, `best`.
To configure gzip with the fastest compression (so, bigger size): deviceinfo_initfs_compression="gzip:fast"`
To configure zstd with the most compression: `deviceinfo_initfs_compression="zstd:best"`
To configure zstd with default compression: `deviceinfo_initfs_compression="zstd"` (or `deviceinfo_initfs_compression="zstd:default"`)
In this case, `gzip:default` is assumed: deviceinfo_initfs_compression="bananas:mmmm"`
There's really not a great way to map individual levels to each
compression library, so this just adds a new type that will invoke the
three relevant levels for each library used. This could be improved in
the future.
This replaces the parallel gzip with the boring gzip from Go's standard
lib. The main motivations for doing this are:
1) Reduce runtime memory requirements
2) shed some external dependencies
There's obviously a trade-off with compression speed/time (as seen
below), but I feel like it's a worthwhile trade-off.
Note that there's likely very little impact to boot performance wrt
extracting these archives, the compression levels are similar.
Measured on a Shift 6mq, which is a very fast phone...
** compress/gzip:
User time (seconds): 1.81
System time (seconds): 0.38
Percent of CPU this job got: 104%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.09
Maximum resident set size (kbytes): 62024
-rw-r--r-- 1 clayton clayton 6.1M Sep 20 17:20 initramfs
-rw-r--r-- 1 clayton clayton 2.5M Sep 20 17:20 initramfs-extra
** pgzip:
User time (seconds): 1.19
System time (seconds): 0.48
Percent of CPU this job got: 159%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.05
Maximum resident set size (kbytes): 109952
-rw-r--r-- 1 clayton clayton 6.8M Sep 20 17:20 initramfs
-rw-r--r-- 1 clayton clayton 2.8M Sep 20 17:20 initramfs-extra
inspired by: https://gitlab.com/postmarketOS/pmaports/-/issues/1704
Checking for the app is unnecessary, since the cmd.Run later will fail
if it doesn't exist. This allows dropping the dependency on misc.Exists.
There's also no reason to print that the command failed, just return the
error.
