When debugging, particularly when stepping through code in a debugger or
dealing with very slow operations, the console timeout can interfere.
Add a --no-timeout command-line option that disables the console
timeout. Adjust get_default_timeout() to checks for both --gdbserver and
--no-timeout, returning None to disable timeouts in either case. This
consolidates the timeout-disable logic that was previously spread across
multiple locations.
Series-to: concept
Series-cc: heinrich
Cover-letter:
Malloc debugging and test/py improvements
This series adds malloc-debugging features including a traffic log and
file output for dumps, along with video optimisations and test/py
performance improvements.
The overall goal is to speed up pytests and make it easier to debug
memory leaks.
Changes include:
- Sandbox gprof profiling and mcheck runtime-disable support
- Video truetype scratch buffer to reduce malloc pressure
- Malloc traffic logging with 'malloc log' command
- File output for malloc dump and log
- test/py performance improvements reducing CPU usage by ~30%
END
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Series-links: 1:100
Series-version: 2
Add a --no-full option to pytest that passes -F to sandbox, skipping
flat-tree tests. This allows running only live-tree tests for faster
iteration during development.
The default is to run full tests (both live and flat tree). Use
--no-full to run only live-tree tests.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The VT100 escape sequence filter is applied to the entire buffer on
every read iteration, causing O(n^2) performance. With 1.6MB of output
from verbose unit tests and 1KB reads, this results in ~1600 iterations,
each processing an ever-growing buffer.
Fix this by filtering only the newly received data before appending it
to the buffer.
To test this, dm_test_host() was modified to do 10 malloc_dump() calls,
thus producing a lot of output. The impact of this patch is:
total time 6s -> 4.5s
total CPU 18s -> 13s
Note: Various other approaches were tried, including removing the
concatenation to self.buf and updating expect() to only consider recent
text. None of these yielded useful results.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
In some cases the target sends a lot of data. This is typically limited
to 4K on Linux (PTY size), but when (say) 100K of console output is
sent, pytest uses all available CPU, often only reading 50 bytes at a
time.
Add a 1ms delay before polling, to help with this. Increase the
read-buffer size from 1KB to 4KB to reduce the number of system calls.
To test this change, dm_test_host() was modified to do 10 malloc_dump()
calls, thus producing a lot of output. The impact of this patch is:
total time 17s -> 6s
total CPU 40s -> 18s
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
If the build does not exist we can get an error when starting up:
INTERNALERROR> Exception: .config does not exist; try passing --build
option?
Exception ignored in atexit callback: <function cleanup at ...>
Traceback (most recent call last):
File "test/py/conftest.py", line 697, in cleanup
show_timings()
File "test/py/conftest.py", line 644, in show_timings
if ubconfig and ubconfig.timing:
^^^^^^^^^^^^^^^
Fix this by setting up the timing member at the start.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add malloc_log_dump() to print all recorded malloc/free/realloc calls
with their addresses, sizes, and caller information. This provides a
way to inspect the log after recording.
The dump shows a summary line with entry counts, followed by a table
with sequence number, operation type, pointer, size, and caller for
each recorded operation.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a malloc()-traffic log that records all malloc()-related calls with
their addresses, sizes, and caller information. This is useful for
debugging allocation patterns and finding the source of allocations that
lack caller info in heap dumps.
Each entry stores:
- Operation type (alloc/free/realloc/memalign)
- Pointer address
- Size (and old size for realloc)
- Full caller backtrace string
On sandbox, the log buffer is allocated from host memory using
os_malloc(), so it does not affect U-Boot's heap. The size is
controlled by CONFIG_MCHECK_LOG_SIZE (default 512K entries).
If the log fills up, it wraps around (circular buffer) and a warning
is shown when dumping to indicate how many entries were lost.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The malloc_very_large() test fails when mcheck is enabled with large
CONFIG_MCHECK_CALLER_LEN because the 64K margin does not account for
the per-allocation overhead (header + canaries).
Use a larger margin (256K) when mcheck is enabled to ensure the test
passes regardless of the mcheck caller length setting.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a _base suffix to this test so that it is easier to run it by
itself with test.py without also getting dm_test_host_dup()
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The run_test_nand() function allocates buf and gold buffers but never
frees them, leaking about 2MB per test run.
Fixes: bc8e8a4bfa ("nand: Add sandbox driver")
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The ext4l driver supports metadata_csum checksums, unlike the old ext4
driver. Only disable metadata_csum when CONFIG_FS_EXT4L is not enabled,
allowing ext4l to use modern ext4 filesystem features.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add ext4l_rename() to rename files and directories, including moves
across directories. This uses the Linux ext4_rename() function.
Also fix the symlink test to verify reading through symlinks works
correctly, since ext4l_resolve_path follows symlinks (stat behavior).
Add Python test wrappers for mkdir, ln, and rename tests.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The ext4l_ln() function returned -EEXIST when creating a symlink where
a file already exists. This differs from the old ext4 implementation
which deletes any existing file before creating the symlink (like ln -sf
behaviour).
Update ext4l_ln() to match this behaviour by calling __ext4_unlink() to
remove any existing non-directory file before creating the symlink.
Directories cannot be replaced with symlinks and return -EISDIR.
This allows test_symlink3 to pass.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add ext4l_ln() to create symbolic links. This uses the Linux ext4_symlink()
function which supports both fast symlinks (stored in inode) and regular
symlinks (stored in data blocks).
Fix the fscrypt_prepare_symlink() stub to properly init the disk_link
structure with the symlink target, which is required for symlink creation
to work correctly.
Add some notes about U-Boot's argument ordering with symlinks.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Implement ext4l_mkdir() to create directories on ext4 filesystems.
The function parses the path to extract the parent directory and
basename, resolves the parent inode, checks for existing entries,
and calls the Linux ext4_mkdir() function to create the directory.
Hook ext4l_mkdir into the filesystem layer via the .mkdir callback
in fs_legacy.c, enabling the standard 'mkdir' command to work with
ext4l filesystems.
Add a unit test that verifies directory creation, duplicate detection
(-EEXIST), nested directory creation, and error handling for
non-existent parent directories (-ENOENT).
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Implement ext4l_unlink() to delete files from ext4 filesystems. This
enables the 'rm' command to work with ext4l.
The implementation:
- Resolves the parent directory and target file
- Verifies the target is not a directory (use rmdir for that)
- Calls ext4_unlink() to remove the directory entry
- Uses journal transactions for crash safety
Add ext4l_op_ptr() macro to select between ext4l_unlink() and the
fallback based on CONFIG_EXT4_WRITE
Call ext4_commit_super() to ensure the changes are written to disk.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add the ability to write files to ext4 filesystems using the ext4l
driver. This enables the 'save' command to work with ext4l.
The implementation uses the jbd2 journal for crash safety:
- ext4l_write() creates files if needed and writes data
- Journal transactions commit synchronously for durability
- Buffer cache syncs dirty buffers after write operations
The write path consists of the following steps:
1. Lookup or create file via ext4_create()
2. Start journal transaction
3. For each block: get/allocate block, copy data, sync to disk
4. Update inode size and commit transaction
5. Sync all dirty buffers
Add an ext4l_op_ptr() macro to select between a write operation and a
fallback based on CONFIG_EXT4_WRITE, avoiding #ifdefs in fstypes[].
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When running sandbox with lots of console output, the vidconsole slows
things down significantly since it renders truetype fonts to the
internal framebuffer. Add a -Q/--quiet_vidconsole command-line option
that removes vidconsole from stdout and stderr, using only the serial
console.
This can provide a ~300x speedup for output-heavy operations.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Split the base_script to separate the ut command from the U-Boot command
sequence. This allows using run_ut() for the vbe_test_fixup test,
reducing duplication and improving consistency.
Cover-letter:
test: py: Convert more tests to use run_ut() helper
This series converts additional Python tests to use the run_ut() helper.
This reduces duplication and makes the tests more concise and consistent.
The ext4l tests are straightforward conversions, while test_vbe.py
requires splitting the U-Boot command script to separate the ut command.
END
Series-to: concept
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Convert test_ext4l.py to use the run_ut() helper instead of manually
building the ut command and checking for failures. This reduces
duplication and makes the tests more concise.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Running the full test suite takes a long time. It would be useful to
distribute tests across multiple sandbox instances to speed up testing.
Add support for running tests in parallel across multiple sandbox
instances. Each worker runs a subset of tests based on its worker ID.
Add -P<n>:<w> option to the ut command where n is the total number of
workers and w is this worker's ID (0 to n-1). Tests are distributed
by index modulo number of workers.
Series-to: u-boot
Series-cc: heinrich
Cover-letter:
test: Various improvements to unit-test infrastructure
This series adds several improvements to the unit-test infrastructure:
- Move disk images to the persistent-data directory so they don't
pollute the source tree
- Add a way to keep pytest-created artefacts for faster iteration on
C tests
- Add a helper to simplify running manual unit tests from Python
- Allow combined flags with the ut command (e.g. -Efm)
- Add a -E flag to emit machine-readable result lines
- Add a -P flag to distribute tests across parallel workers
- Add -m as an alias for -f (force manual tests)
These changes make it easier to run and debug tests, especially when
iterating on C test code.
END
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The ut command shows test output but does not provide a machine-readable
indication of whether each individual test passed or failed. External
tools must rely on heuristics like scanning for failure patterns in the
output.
Add a -E flag that emits an explicit result line after each test:
Result: PASS: test_name: file.c
Result: FAIL: test_name: file.c
This allows tools to reliably determine per-test pass/fail status
without fragile pattern matching. The flag is optional to maintain
backward compatibility with existing scripts.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The current argument-parsing logic uses switch (str[1]) which only
processes the second character of each argument. This prevents combining
multiple single-character flags in one argument (e.g., -fm).
Refactor the code to use a for loop that iterates through all characters
in the argument. For flags that take a value (like -r and -I), use goto
to skip the rest of the argument after processing.
This allows combined flags like -fmR instead of requiring -f -m -R.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add -m as an alias for the -f flag which forces manual tests to run.
This provides consistency with external test runners that will use -m
for "manual" tests.
Also update the documentation to explain what manual tests are, and fix
a typo ("types" -> "times") in the -r description.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Running manual unit tests (those with _norun suffix) involves a common
pattern: building the ut command with the -f flag, running it, and
checking for failures. This is verbose and error-prone.
Add a run_ut() method to ConsoleBase that simplifies this. It handles
the command construction, test arguments and failure checking
automatically.
Before:
output = ubman.run_command(
f'ut -f fs fs_test_ext4l_probe_norun fs_image={ext4_image}')
assert 'failures: 0' in output
After:
ubman.run_ut('fs', 'fs_test_ext4l_probe', fs_image=ext4_image)
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When iterating on C test code, the Python fixtures that create disk
images run each time, even though the images have not changed. This
slows down the development cycle unnecessarily.
Add a -P/--persist option to prevent cleanup of test-generated files
like disk images. This allows re-running C tests directly, without
re-running the Python fixture each time.
Update the ext4l test to respect the persist flag.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
It is annoying to have disk images in the source directory since it
clutters up the working space.
Move spi.bin (used by the SPI tests) into the persistent-data
directory, update the driver and add a comment.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
It is annoying to have disk images in the source directory since it
clutters up the working space.
Remove cur_dir=True from DiskHelper calls so disk images are written to
the persistent-data directory instead.
Move scsi.img too (used by the bootstd tests) and mmc6.img (used by the
MBR tests.
Add a few comments as to where the images are used.
This keeps the source tree clean and puts disk images in the same place
as other test data.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Implement ext4l_statfs() to return filesystem statistics using the
ext4 superblock. The function returns block size, total block count,
and free block count.
Add unit test to verify the statfs implementation returns valid data.
Cover-letter:
fs: ext4l: Complete read-only filesystem support (Part I)
This series completes read-only support for the ext4l filesystem driver,
which is a port of the Linux ext4 driver to U-Boot.
The ext4l driver provides more complete ext4 support than the existing
ext4 driver, including proper handling of extents, directory hashing,
and other ext4 features.
Changes include:
Sandbox infrastructure:
- Fix IRQ macros and buffer_head includes for sandbox builds
Core fixes:
- Fix path lookup by implementing proper dentry operations
- Fix fscrypt_match_name to do actual name comparison
Filesystem operations:
- Add directory listing (opendir/readdir/closedir)
- Add file existence check (exists)
- Add file size query (size)
- Add file read support (read)
- Add UUID query (uuid)
- Add filesystem statistics (statfs)
New command:
- Add fsinfo command to display filesystem statistics
Testing:
- Add comprehensive unit tests for all operations
- Enable fsuuid command for sandbox testing
END
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add the fsinfo command to display filesystem statistics including block
size, total blocks, used blocks, and free blocks. Both raw byte counts
and human-readable sizes are shown.
Example output:
=> fsinfo mmc 0:1
Block size: 4096 bytes
Total blocks: 16384 (67108864 bytes, 64 MiB)
Used blocks: 2065 (8458240 bytes, 8.1 MiB)
Free blocks: 14319 (58650624 bytes, 55.9 MiB)
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The filesystem uuid method is not implemented.
Add ext4l_uuid() which returns the filesystem UUID as a string and
wire it into the filesystem operations table. Add a test.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add ext4l_read() function to read file contents. The function resolves
the path to an inode, then reads the file block by block using
ext4_bread() and copies the data to the output buffer.
Signed-off-by: Simon Glass <sjg@chromium.org>
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add ext4l_size() function to retrieve the size of a file or directory.
Wire it into the filesystem operations table in fs_legacy.c.
Signed-off-by: Simon Glass <sjg@chromium.org>
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Implement ext4l_exists() to check if a file or directory exists on the
filesystem. This uses ext4l_resolve_path() to look up the path and
returns 1 if found, 0 otherwise.
Wire the function into fs_legacy.c and add a basic test.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Implement directory-iteration for the ext4l filesystem driver, allowing
callers to iterate through directory entries one at a time.
Add ext4l_opendir() which opens a directory and returns a stream handle,
ext4l_readdir() which returns the next directory entry, and
ext4l_closedir() which closes the stream and frees resources.
The implementation uses a struct dir_context to capture single entries
from ext4_readdir(), with logic to skip previously returned entries
since the htree code may re-emit them.
Update struct file to include a position.
Wire these functions into fs_legacy.c for the ext4l filesystem type.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Directory entries appear in hash order and mount messages may appear
multiple times during probe operations.
Update tests to skip remaining output after finding expected content.
We could perhaps refine this with more powerful matching, but that is
left for another day.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Provide test environments for running
qemu-riscv64_smode and qemu-riscv64_smode_acpi.
These use the OpenSBI implementation provided by QEMU and
pass main u-boot as -kernel parameter.
ACPI is enabled for both boards. We still expect the boards to use
QEMU's device-tree as control device-tree.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
(cherry picked from d94ebdd2c823ce8776723184cda960dcd2954711)
Remove the rva23s64 CPU since we don't have that in our QEMU:
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a --quiet-hooks option to suppress display of hook commands to
stdout while still logging them to the HTML log file. This reduces
console noise during test runs.
Cover-letter:
test: pytest and hook improvements
This series includes several improvements to the pytest infrastructure
and test hook configurations:
- Fix the timing check in conftest to properly check if timing is enabled
- Add a --quiet-hooks option to suppress hook command display
- Add and update ellesmere hook config files for various QEMU boards
- Update qemu-riscv64 config for S-mode with OpenSBI
END
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add configuration files for qemu-riscv32_smode, qemu-riscv64_smode,
qemu-riscv64_smode_acpi, qemu_arm64_acpi, and qemu_arm64_lwip boards.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Update the qemu-riscv64 hook config to use OpenSBI as the BIOS and load
U-Boot as the kernel, which is required for S-mode operation.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add configuration files for qemu-arm-sbsa, qemu-ppce500, and
qemu-xtensa-dc233c boards.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The check uses hasattr() to see if the 'timing' attribute exists, but
does not check if it is actually enabled. Fix this by checking the
attribute value instead.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Should use CONFIG_64BIT to detect a 64 bit compile and not
CONFIG_PHYS_64BIT. This allows more platforms to run the
full test code.
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Andrew Goodbody <andrew.goodbody@linaro.org>
(cherry picked from commit 43ca65b305)
lmb_alloc_base() is just calling lmb_alloc_base_flags() with LMB_NONE.
There's not much we gain from this abstraction, so let's remove the
former add the flags argument to lmb_alloc_base() and make the code
a bit easier to follow.
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Tested-by: Sam Protsenko <semen.protsenko@linaro.org>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
(cherry picked from commit 3075708017)
lmb_alloc_addr() is just calling lmb_alloc_addr_flags() with LMB_NONE
There's not much we gain from this abstraction, so let's remove the
latter, add a flags argument to lmb_alloc_addr() and make the code a
bit easier to follow.
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Tested-by: Sam Protsenko <semen.protsenko@linaro.org>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
(cherry picked from commit 15e0c5e390)
free_mem is a misnomer. We never update it with the free memory for
LMB. Instead, it describes all available memory and is checked against
used_mem to decide whether an area is free or not.
So let's rename this field to better match its usage.
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Tested-by: Sam Protsenko <semen.protsenko@linaro.org>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
(cherry picked from commit 400c34db89)
lmb_reserve() is just calling lmb_reserve_flags() with LMB_NONE.
There's not much we gain from this abstraction.
So let's remove the latter, add the flags argument to lmb_reserve()
and make the code a bit easier to follow.
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Tested-by: Sam Protsenko <semen.protsenko@linaro.org>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
(cherry picked from commit 900a8951c3)
LMB flags is not an enum anymore. It's currently used as a bitmask
in various places of our code. So make it a u32 which is more
appropriate when dealing with masks.
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Tested-by: Sam Protsenko <semen.protsenko@linaro.org>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
(cherry picked from commit 3d56c06551)
