When running sandbox with lots of console output, the vidconsole slows
things down significantly since it renders truetype fonts to the
internal framebuffer. Add a -Q/--quiet_vidconsole command-line option
that removes vidconsole from stdout and stderr, using only the serial
console.
This can provide a ~300x speedup for output-heavy operations.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When os_persistent_file() is called without a directory set in the
environment variable and the file isn't found in the current directory,
also check in the executable's directory.
This allows tests like dm_test_host to work when run directly from the
build directory rather than through the pytest framework, avoiding the
need to set U_BOOT_PERSISTENT_DATA_DIR manually.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Rename local variables in cmpxchg macro to avoid shadowing when used
inside try_cmpxchg, which also declares __old. Clang complains about
"variable '__old' is uninitialised when used within its own
initialisation" due to the nested macro expansion.
Cover-letter:
ext4l: Infrastructure and fixes for write support (part K)
This series adds infrastructure and bug fixes needed for ext4l write
support. It includes:
- kmem_cache implementation controlled by CONFIG_LIB_KMEM_CACHE
- Bit operation functions imported from Linux (find_bit, fns)
- Little-endian bit operations for ext4 bitmaps
- Buffer I/O infrastructure for write operations
- Folio and buffer head fixes for U-Boot's malloc'd buffers
- Inode handling fixes (i_mode, i_blocks, iput eviction)
- Journal cleanup detection in bh_cache_clear()
- Various bug fixes for clang warnings and multi-word bit operations
END
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The set_bit(), clear_bit(), and change_bit() functions only modify the
first word of a bitmap, regardless of the bit number. For bit numbers
>= 64 on 64-bit systems, the shift wraps around and modifies the wrong
bit position.
Fix these functions to properly calculate the word offset before
modifying the bit. This is needed for block bitmap operations where bit
numbers can be in the thousands.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Update the percpu_counter_initialized() and percpu_counter_init()
macros in ext4_uboot.h to use the new initialized field.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Implement iput() to properly handle inode reference counting and
eviction. When the reference count drops to zero and the inode has no
links (i_nlink == 0), call ext4_evict_inode() to free the inode's data
blocks and the inode itself.
This is required for unlink operations to properly free filesystem
resources.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The ktime_get_real_seconds() stub returns 0, causing deleted inodes
to have zero i_dtime. This causes fsck to complain about deleted
inodes with zero deletion time.
Instead, use the boot-relative time in seconds. While not a real
wall-clock timestamp, it provides a non-zero value that satisfies
filesystem-consistency checks.
Future work can improve on this, perhaps using an on-board RTC.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The dquot_alloc_block(), dquot_free_block() and dquot_alloc_block_nofail()
functions are stubs that do nothing. These functions are called by ext4
when allocating and freeing blocks, and they should update the inode's
i_blocks field.
Fix these functions to properly track block allocation in i_blocks,
which is stored in 512-byte units.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add declaration for ext4_commit_super() to the ext4_uboot.h header
and update the comment to reflect both superblock initialisation
and commit functions.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Fix inode_init_owner() to properly set i_mode, which is needed for
ext4_create() to work correctly.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add more complete buffer I/O infrastructure so that we can support
writing to an ext4 filesystem.
- end_buffer_write_sync(): I/O completion callback for sync writes
- submit_bh(): Add the write path with b_end_io callback invocation
- __getblk(): Allocate journal-descriptor buffers
- free_buffer_head(): Don't free shared folios from shadow buffers
- __brelse(): Only decrement refcount, don't free
- bh_cache_sync(): Sync all dirty buffers to disk
- sync_dirty_buffer()/mark_buffer_dirty(): Write buffer immediately
The shadow-buffer fix is needed for jbd2 journaling: shadow buffers
created by jbd2_journal_write_metadata_buffer() share their folio with
the original buffer (as indicated by b_private).
With this, the buffer I/O layer is ready for ext4 write support.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The existing folios macros assume page-aligned memory, but U-Boot uses
malloc'd buffers for simplicity.
Update the macros accordinging:
- offset_in_folio(): Calculate the offset from the folio's data pointer
- bh_offset(): Calculate the actual offset within the folio
- folio_set_bh(): Actually set b_folio and b_data
- kmap_local_folio(): Return a pointer to folio data + offset
Implement __filemap_get_folio(), folio_put() and folio_get() for
folio-lifecycle management.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The ext4 block allocator uses little-endian bit operations on block
bitmaps. Implement these operations by wrapping the existing
set/test/clear_bit() functions.
Add find_next_zero_bit() to search for free blocks in bitmaps.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add lib/find_bit.c and include/linux/find.h from Linux v6.19, trimmed
to include only the functions needed for ext4l: find_first_bit(),
find_first_zero_bit(), find_next_bit(), find_next_zero_bit() and
find_last_bit()
The following items are removed from the Linux originals:
- find.h: _and_bit, _andnot_bit, _or_bit, _nth_bit variants, wrap
functions, clump8 functions, big-endian support, most for_each_...
macros
- find_bit.c: Corresponding implementations, random.h include
Add wrapper functions matching sandbox's asm/bitops.h declarations
(int return type, void* addr) that call the _find_* implementations.
Build find_bit.o only for sandbox for now.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add the fns() function from Linux to find the N'th set bit in a word.
This is needed by lib/find_bit.c which uses it in the FIND_NTH_BIT
macro.
Taken from Linux v6.19
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When debugging journal-cleanup issues, stale journal_head attachments on
buffer_heads can cause crashes on subsequent mounts.
Add detection logic in bh_cache_clear() to warn when a buffer_head still
has a journal_head attached. This indicates the journal was not properly
destroyed before unmount. Clear the JBD flag and pointer to prevent
issues with subsequent mounts.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a Kconfig option to control whether full kmem_cache_free() and
kmem_cache_destroy() implementations are provided in lib/linux_compat.c
Most boards do not need these functions, so they can use simple inline
stubs in slab.h. Subsystems like ext4 that require proper cache
management can select CONFIG_LIB_KMEM_CACHE.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Initialize 'len' to 0 to silence a Clang warning about the potential use
of an uninitialised variable. The else branch calls BUG() which never
returns, but Clang cannot determine this.
Series-to: concept
Cover-letter:
ext4l: Linux adaptation patches for ext4 write support
This series contains adaptations to Linux-imported files needed for
ext4l write support in U-Boot. These changes are separated from the
main ext4l implementation to make it easier to track modifications
to imported code.
The patches include:
- Bit position fixes for REQ_OP and BH_OwnsData to avoid conflicts
- JBD2 journal adaptations for U-Boot's single-threaded environment
- Function exports to allow calling ext4 internals from U-Boot code
- Cache management fixes for multiple mount/unmount cycles
- Compiler warning fixes for Clang compatibility
These changes are minimal modifications to the Linux ext4 and jbd2
code, using #ifdef __UBOOT__ guards where appropriate to ease future
Linux updates.
END
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Series-links: 1:96
In U-Boot, filesystems may be mounted and unmounted multiple times in a
single session. The ext4 cache initialization functions would fail on
subsequent mounts because the caches were already initialized but pointers
were not reset on exit.
Add early return checks in init functions when already initialized, and
reset cache pointers to NULL in exit functions to allow clean
reinitialization.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Remove static from ext4_commit_super() to allow calling it from
ext4l interface code to sync superblock after write operations.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add __maybe_unused to ext4_groupinfo_slab_names since U-Boot's
kmem_cache_create macro ignores the name parameter, making the array
appear unused to the compiler.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
In U-Boot, filesystems may be mounted and unmounted multiple times in a
single session. The JBD2 global state (caches) was only initialized once
and never cleaned up, preventing proper reinitialization.
Add jbd2_journal_exit_global() to properly destroy caches and reset the
initialization flag. This allows the JBD2 subsystem to be cleanly
reinitialized on subsequent mounts.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When debugging journal corruption issues, invalid journal_head or
buffer_head pointers can cause crashes that are difficult to diagnose.
Add explicit validation of jh_in and its associated buffer_head at the
start of jbd2_journal_write_metadata_buffer() to catch corruption early
and provide useful debug output rather than crashing with a SIGSEGV.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
In U-Boot's ext4l implementation, bh_cache_sync() writes all dirty
buffers to disk. However, buffers passed through the journal via
jbd2_journal_dirty_metadata() were not being marked dirty for
bh_cache_sync() to pick up.
Add mark_buffer_dirty() after jbd2_journal_dirty_metadata() to ensure
these buffers are written by bh_cache_sync(). This is needed because
U-Boot's journal implementation does not write buffers to their final
locations.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add an 'initialized' field to struct percpu_counter to track whether
a counter has been properly initialized. Update percpu_counter_init()
to set this field and percpu_counter_initialized() to check it.
This is needed because ext4 uses percpu_counter_initialized() to check
if counters are ready before accessing them.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a simple folio cache array to struct address_space for U-Boot's
folio management, avoiding the need for Linux's XArray/radix tree
infrastructure.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Skip orphan handling in U-Boot. We do synchronous journal commits
after each operation, so orphan recovery is not needed. Adding to
the orphan list without proper locking can corrupt the list.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Remove static from ext4_create(), ext4_mkdir(), ext4_symlink(), and
ext4_rename2() and add declarations in ext4.h to allow calling them
from ext4l interface code.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
U-Boot operates in a single-threaded environment without a journal
daemon. Commit transactions synchronously when jbd2_journal_stop()
is called and there are no active handles (t_updates == 0).
This ensures crash-safety by writing journal entries to disk immediately
after each file-operation completes.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The REQ_OP flags (REQ_SYNC, REQ_FUA) use bits 0-1, which collide
with REQ_OP_WRITE (value 1). Move the flags to bits 8+ and add
REQ_OP_MASK to properly separate operation from flags.
Move BH_OwnsData from buffer_head.h to ext4_uboot.h to keep
Linux headers unmodified. Define it as BH_JBDPrivateStart to
avoid conflicts with JBD2 state bits (17-25).
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Split the base_script to separate the ut command from the U-Boot command
sequence. This allows using run_ut() for the vbe_test_fixup test,
reducing duplication and improving consistency.
Cover-letter:
test: py: Convert more tests to use run_ut() helper
This series converts additional Python tests to use the run_ut() helper.
This reduces duplication and makes the tests more concise and consistent.
The ext4l tests are straightforward conversions, while test_vbe.py
requires splitting the U-Boot command script to separate the ut command.
END
Series-to: concept
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Convert test_ext4l.py to use the run_ut() helper instead of manually
building the ut command and checking for failures. This reduces
duplication and makes the tests more concise.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Running the full test suite takes a long time. It would be useful to
distribute tests across multiple sandbox instances to speed up testing.
Add support for running tests in parallel across multiple sandbox
instances. Each worker runs a subset of tests based on its worker ID.
Add -P<n>:<w> option to the ut command where n is the total number of
workers and w is this worker's ID (0 to n-1). Tests are distributed
by index modulo number of workers.
Series-to: u-boot
Series-cc: heinrich
Cover-letter:
test: Various improvements to unit-test infrastructure
This series adds several improvements to the unit-test infrastructure:
- Move disk images to the persistent-data directory so they don't
pollute the source tree
- Add a way to keep pytest-created artefacts for faster iteration on
C tests
- Add a helper to simplify running manual unit tests from Python
- Allow combined flags with the ut command (e.g. -Efm)
- Add a -E flag to emit machine-readable result lines
- Add a -P flag to distribute tests across parallel workers
- Add -m as an alias for -f (force manual tests)
These changes make it easier to run and debug tests, especially when
iterating on C test code.
END
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add documentation for the return value of the ut command:
- Returns 0 on success if all tests pass
- Returns 1 on failure if any test fails
- Skipped tests do not cause a failure
Also explain when tests may be skipped and how to detect skipped tests
programmatically using the -E flag.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The ut command shows test output but does not provide a machine-readable
indication of whether each individual test passed or failed. External
tools must rely on heuristics like scanning for failure patterns in the
output.
Add a -E flag that emits an explicit result line after each test:
Result: PASS: test_name: file.c
Result: FAIL: test_name: file.c
This allows tools to reliably determine per-test pass/fail status
without fragile pattern matching. The flag is optional to maintain
backward compatibility with existing scripts.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The current argument-parsing logic uses switch (str[1]) which only
processes the second character of each argument. This prevents combining
multiple single-character flags in one argument (e.g., -fm).
Refactor the code to use a for loop that iterates through all characters
in the argument. For flags that take a value (like -r and -I), use goto
to skip the rest of the argument after processing.
This allows combined flags like -fmR instead of requiring -f -m -R.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add -m as an alias for the -f flag which forces manual tests to run.
This provides consistency with external test runners that will use -m
for "manual" tests.
Also update the documentation to explain what manual tests are, and fix
a typo ("types" -> "times") in the -r description.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Running manual unit tests (those with _norun suffix) involves a common
pattern: building the ut command with the -f flag, running it, and
checking for failures. This is verbose and error-prone.
Add a run_ut() method to ConsoleBase that simplifies this. It handles
the command construction, test arguments and failure checking
automatically.
Before:
output = ubman.run_command(
f'ut -f fs fs_test_ext4l_probe_norun fs_image={ext4_image}')
assert 'failures: 0' in output
After:
ubman.run_ut('fs', 'fs_test_ext4l_probe', fs_image=ext4_image)
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When iterating on C test code, the Python fixtures that create disk
images run each time, even though the images have not changed. This
slows down the development cycle unnecessarily.
Add a -P/--persist option to prevent cleanup of test-generated files
like disk images. This allows re-running C tests directly, without
re-running the Python fixture each time.
Update the ext4l test to respect the persist flag.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
It is annoying to have disk images in the source directory since it
clutters up the working space.
Move spi.bin (used by the SPI tests) into the persistent-data
directory, update the driver and add a comment.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
It is annoying to have disk images in the source directory since it
clutters up the working space.
Remove cur_dir=True from DiskHelper calls so disk images are written to
the persistent-data directory instead.
Move scsi.img too (used by the bootstd tests) and mmc6.img (used by the
MBR tests.
Add a few comments as to where the images are used.
This keeps the source tree clean and puts disk images in the same place
as other test data.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
This message is not needed since sphinx is now running in quiet mode.
Series-to: concept
Series-cc: heinrich
Cover-letter:
doc: Silence the sphinx build output
The sphinx/htmldocs build is very noisy, making it hard to see warnings
and errors. This series enables quiet mode for sphinx and removes
various other progress messages, so that only warnings and errors are
shown.
END
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Series-links: 1:93
The "enabling CJK for LaTeX builder" message adds noise to the build
output. Remove the print statement since the functionality is still
enabled.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
