Commit Graph

17 Commits

Author SHA1 Message Date
Clayton Craft
7c2377d0c8 archive: add "none" compression format (MR 25) 2023-02-26 12:18:27 -08:00
Clayton Craft
f24d0139c9 archive.writeCompressed: bubble up error when closing archive file (MR 25)
This also prevents a double Close() if the compressor is ever set to 'fd'
2023-02-26 12:17:58 -08:00
Clayton Craft
786e09d855 archive: support using lzma (MR 25)
fixes: https://gitlab.com/postmarketOS/postmarketos-mkinitfs/-/issues/2
2023-02-24 09:21:24 -08:00
Clayton Craft
ba1e1a77db archive: Use compression level when generating archive (MR 25) 2023-02-24 09:21:24 -08:00
Clayton Craft
fd11f4a627 archive.New: accept compression level (MR 25)
I went with a simpler implementation that uses Go compression packages
to do the work. The downside of this is that the compression Level is a
bit weird to set, since most libraries discourage setting the numeric
compression level directly.

This is configured by setting `deviceinfo_initfs_compression`, the value
it expects is a string in the form: `FORMAT[:LEVEL]`, where `[:LEVEL]`
is optional. Actually setting the variable at all is optional... if
nothing is specified, or it can't parse the format/level from the string
value, it defaults to using gzip with the "default" level for the
package (which tries to mirror gzip's default, or something).

The level can be one of `default`, `fast`, `best`.

To configure gzip with the fastest compression (so, bigger size): deviceinfo_initfs_compression="gzip:fast"`

To configure zstd with the most compression: `deviceinfo_initfs_compression="zstd:best"`

To configure zstd with default compression: `deviceinfo_initfs_compression="zstd"` (or `deviceinfo_initfs_compression="zstd:default"`)

In this case, `gzip:default` is assumed: deviceinfo_initfs_compression="bananas:mmmm"`
2023-02-24 09:21:23 -08:00
Clayton Craft
322d6bb754 archive: add ExtractFormatLevel function (MR 25)
Extracts the format and level from a string in the form "format:level"
2023-02-24 09:21:23 -08:00
Clayton Craft
1f4d8737e8 archive: add CompressLevel type and consts (MR 25)
There's really not a great way to map individual levels to each
compression library, so this just adds a new type that will invoke the
three relevant levels for each library used. This could be improved in
the future.
2023-02-24 09:21:23 -08:00
Clayton Craft
52fc741ba8 archive: support using zstd (MR 25)
This external module was chosen because it's a native Go implementation
of zstd, and not a wrapper around some external utility or some CGO
thing.
2023-02-24 09:21:23 -08:00
Clayton Craft
31b7eb34ee archive: allow selecting different formats when writing compressed file (MR 25) 2023-02-24 09:21:23 -08:00
Clayton Craft
4e97990804 archive: accept compression format parameter during instantiation (MR 25) 2023-02-24 09:21:23 -08:00
Clayton Craft
c01b48ad25 archive: add CompressFormat type with initial constants (MR 25) 2023-02-24 09:21:22 -08:00
Clayton Craft
6aec4d564c archive: remove pgzip, use gzip from go std lib (MR 25)
This replaces the parallel gzip with the boring gzip from Go's standard
lib. The main motivations for doing this are:

1) Reduce runtime memory requirements

2) shed some external dependencies

There's obviously a trade-off with compression speed/time (as seen
below), but I feel like it's a worthwhile trade-off.

Note that there's likely very little impact to boot performance wrt
extracting these archives, the compression levels are similar.

Measured on a Shift 6mq, which is a very fast phone...

** compress/gzip:

User time (seconds): 1.81
System time (seconds): 0.38
Percent of CPU this job got: 104%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.09
Maximum resident set size (kbytes): 62024

-rw-r--r-- 1 clayton clayton 6.1M Sep 20 17:20 initramfs
-rw-r--r-- 1 clayton clayton 2.5M Sep 20 17:20 initramfs-extra

** pgzip:

User time (seconds): 1.19
System time (seconds): 0.48
Percent of CPU this job got: 159%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.05
Maximum resident set size (kbytes): 109952

-rw-r--r-- 1 clayton clayton 6.8M Sep 20 17:20 initramfs
-rw-r--r-- 1 clayton clayton 2.8M Sep 20 17:20 initramfs-extra

inspired by: https://gitlab.com/postmarketOS/pmaports/-/issues/1704
2023-02-24 09:21:22 -08:00
Clayton Craft
a7c4fe83ce archive: remove TODO about adding debug mode
moved to an issue: https://gitlab.com/postmarketOS/postmarketos-mkinitfs/-/issues/17
2023-02-21 12:41:18 -08:00
Clayton Craft
0179a0ca5c misc: remove functions now in osutils, update references in project 2023-02-21 12:41:17 -08:00
Clayton Craft
bd09de9232 archive: accept a FileLister in the AddItems method 2023-02-21 00:45:58 -08:00
Clayton Craft
6fdc8937b5 internal/archive: make archiveItem.add private 2023-02-21 00:45:16 -08:00
Clayton Craft
690d008643 internal/{archive,misc}: move from pkgs
These should be internal-only to this project
2023-02-09 13:34:06 -08:00