- Add module-level disable for too-many-lines
- Suppress consider-using-with for NamedTemporaryFile (intentional)
- Change unused linenum to _ in parse_file()
- Suppress too-many-arguments for _check_board()
- Add missing param documentation for parse_extended()
Cover-letter:
buildman: Fix pylint warnings in board.py and boards.py
This series addresses pylint warnings in the buildman board modules,
achieving a 10/10 pylint score.
For board.py, unavoidable warnings (too-many-instance-attributes,
too-few-public-methods, too-many-arguments) are suppressed since the
Board class is a legitimate data container.
For boards.py, the series:
- Fixes line-too-long and missing return documentation warnings
- Refactors complex functions by extracting helper methods to reduce
too-many-branches and too-many-locals warnings
- Adds missing parameter documentation
- Suppresses remaining unavoidable warnings
END
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the board checking logic into a separate static method to reduce
complexity of select_boards()
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the srcdir walk and output file scanning into module-level
functions _check_srcdir_is_current() and _check_output_is_current()
to reduce complexity.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Split the setup logic into _collect_defconfigs() for gathering defconfig
file paths and _start_scanners() for spawning the parallel scan
processes. This reduces complexity and eliminates the too-many-locals
warning.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the handling of F: (file path) and N: (name pattern) tags into
separate class methods _handle_f_tag() and _handle_n_tag(), and target
database updates into _add_targets()
This reduces complexity of parse_file()
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the logic that verifies exactly one TARGET_xxx option is set into
a separate method to reduce complexity of scan()
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the logic that handles #include preprocessing and config loading
into a separate method to reduce complexity of scan()
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the handling of tag lines (name:, desc:, fragment:, targets:) into
a separate method to reduce complexity of parse()
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Change bare except to catch AttributeError specifically, which occurs
when the ARCH_RV32I symbol is not found and get() returns None
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the architecture fixup logic (aarch64 and riscv adjustments) into
a new _fixup_arch() method to reduce complexity of scan().
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Fix line-too-long warnings by wrapping long lines, and add missing
return-type documentation to various functions.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The Board class is a data container that legitimately needs many
attributes and constructor arguments. Suppress the following warnings:
- too-many-instance-attributes
- too-few-public-methods
- too-many-arguments
Also add missing type annotations to the docstring for consistency.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When debugging, particularly when stepping through code in a debugger or
dealing with very slow operations, the console timeout can interfere.
Add a --no-timeout command-line option that disables the console
timeout. Adjust get_default_timeout() to checks for both --gdbserver and
--no-timeout, returning None to disable timeouts in either case. This
consolidates the timeout-disable logic that was previously spread across
multiple locations.
Series-to: concept
Series-cc: heinrich
Cover-letter:
Malloc debugging and test/py improvements
This series adds malloc-debugging features including a traffic log and
file output for dumps, along with video optimisations and test/py
performance improvements.
The overall goal is to speed up pytests and make it easier to debug
memory leaks.
Changes include:
- Sandbox gprof profiling and mcheck runtime-disable support
- Video truetype scratch buffer to reduce malloc pressure
- Malloc traffic logging with 'malloc log' command
- File output for malloc dump and log
- test/py performance improvements reducing CPU usage by ~30%
END
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Series-links: 1:100
Series-version: 2
Add a --no-full option to pytest that passes -F to sandbox, skipping
flat-tree tests. This allows running only live-tree tests for faster
iteration during development.
The default is to run full tests (both live and flat tree). Use
--no-full to run only live-tree tests.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The VT100 escape sequence filter is applied to the entire buffer on
every read iteration, causing O(n^2) performance. With 1.6MB of output
from verbose unit tests and 1KB reads, this results in ~1600 iterations,
each processing an ever-growing buffer.
Fix this by filtering only the newly received data before appending it
to the buffer.
To test this, dm_test_host() was modified to do 10 malloc_dump() calls,
thus producing a lot of output. The impact of this patch is:
total time 6s -> 4.5s
total CPU 18s -> 13s
Note: Various other approaches were tried, including removing the
concatenation to self.buf and updating expect() to only consider recent
text. None of these yielded useful results.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
In some cases the target sends a lot of data. This is typically limited
to 4K on Linux (PTY size), but when (say) 100K of console output is
sent, pytest uses all available CPU, often only reading 50 bytes at a
time.
Add a 1ms delay before polling, to help with this. Increase the
read-buffer size from 1KB to 4KB to reduce the number of system calls.
To test this change, dm_test_host() was modified to do 10 malloc_dump()
calls, thus producing a lot of output. The impact of this patch is:
total time 17s -> 6s
total CPU 40s -> 18s
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
If the build does not exist we can get an error when starting up:
INTERNALERROR> Exception: .config does not exist; try passing --build
option?
Exception ignored in atexit callback: <function cleanup at ...>
Traceback (most recent call last):
File "test/py/conftest.py", line 697, in cleanup
show_timings()
File "test/py/conftest.py", line 644, in show_timings
if ubconfig and ubconfig.timing:
^^^^^^^^^^^^^^^
Fix this by setting up the timing member at the start.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Document the practical workflow for detecting and diagnosing memory
leaks in U-Boot, particularly in sandbox builds. This covers:
- Using ut_check_delta() in unit tests for leak detection
- Comparing heap dumps with malloc_dump_to_file()
- Using malloc traffic logging to trace allocations
- Verifying debug functions don't affect heap state
- A step-by-step practical workflow
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add malloc_dump_to_file() and malloc_log_to_file() functions to write
heap dumps and malloc traffic logs to host files. This is useful for
debugging memory leaks in sandbox by allowing comparison of before/after
heap states with external tools like diff.
These functions are only available in sandbox builds since they use
host-filesystem access.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Backtrace collection is relatively expensive and can significantly slow
down malloc()-heavy code when mcheck is enabled.
Add a new CONFIG_MCHECK_BACKTRACE option (default y) to allow disabling
backtrace collection while keeping the other mcheck features (canaries,
double-free detection, etc.) enabled. This allows using mcheck with less
overhead when caller information is not needed.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add malloc_log_dump() to print all recorded malloc/free/realloc calls
with their addresses, sizes, and caller information. This provides a
way to inspect the log after recording.
The dump shows a summary line with entry counts, followed by a table
with sequence number, operation type, pointer, size, and caller for
each recorded operation.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a malloc()-traffic log that records all malloc()-related calls with
their addresses, sizes, and caller information. This is useful for
debugging allocation patterns and finding the source of allocations that
lack caller info in heap dumps.
Each entry stores:
- Operation type (alloc/free/realloc/memalign)
- Pointer address
- Size (and old size for realloc)
- Full caller backtrace string
On sandbox, the log buffer is allocated from host memory using
os_malloc(), so it does not affect U-Boot's heap. The size is
controlled by CONFIG_MCHECK_LOG_SIZE (default 512K entries).
If the log fills up, it wraps around (circular buffer) and a warning
is shown when dumping to indicate how many entries were lost.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Replace direct printf calls in malloc_dump() with an output callback
function. This introduces dump_out_fn type and dump_to_console() helper,
with malloc_dump_impl() taking the callback and context pointer.
This allows the same implementation logic to be reused for different
output destinations such as writing to a file.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The malloc_very_large() test fails when mcheck is enabled with large
CONFIG_MCHECK_CALLER_LEN because the 64K margin does not account for
the per-allocation overhead (header + canaries).
Use a larger margin (256K) when mcheck is enabled to ensure the test
passes regardless of the mcheck caller length setting.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When mcheck is enabled, malloc_usable_size() returns incorrect results
because mem2chunk() is called on the offset user pointer rather than the
actual chunk.
The pointer returned to the user is offset by the mcheck header, but
malloc_usable_size() is unaware of this. Add a wrapper that returns the
user-requested size stored in the mcheck header. This fixes test
failures when CONFIG_MCHECK_CALLER_LEN is set to larger values.
Also add a wrapper for the case where MALLOC_DEBUG is enabled without
MCHECK_HEAP_PROTECTION, since MALLOC_DEBUG makes
dlmalloc_usable_size_impl() static but no public dlmalloc_usable_size
exists outside the mcheck block.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
In some cases longer function names mean that 48 characters is not
enough to determine the call path. Increase the default to 64 to handle
this.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a _base suffix to this test so that it is easier to run it by
itself with test.py without also getting dm_test_host_dup()
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When copying partial framebuffer regions line by line, there is overhead
from multiple memcpy() calls.
Optimise video_flush_copy() to detect when entire lines are being copied
(damage spans full width) and perform a single memcpy() for the whole
region instead of looping line by line.
Also invert the early-exit check to reduce nesting.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The stb_truetype library performs around 5 allocations per character
rendered, totalling approximately 26KB of temporary memory. This creates
significant malloc/free overhead and heap fragmentation.
Add a scratch buffer mechanism that pre-allocates memory once during
probe and reuses it for each character. The buffer is reset at the start
of each putc_xy() call, and allocations come from this buffer using a
simple bump allocator with 8-byte alignment.
If the scratch buffer is exhausted (e.g. for very complex glyphs), the
allocator falls back to malloc transparently.
The scratch buffer is controlled by two new Kconfig options:
- CONSOLE_TRUETYPE_SCRATCH: Enable/disable the feature (default y)
- CONSOLE_TRUETYPE_SCRATCH_SIZE: Buffer size in bytes (default 32KB)
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
This was last updated in 2023. The updates are minimal but we may as
well keep it up to date.
Bring in the latest version:
f1c79c0 ("Merge pull request #1851 from jeffrbig2/master"_
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a command-line option (-M or --no_mcheck) to disable mcheck heap
protection at runtime. When mcheck is disabled, the wrapper functions
pass through directly to the underlying allocator without adding
headers or checking for corruption.
This is useful for debugging when mcheck interferes with test results,
such as when memory-leak detection reports false positives due to
accumulated allocations from other tests.
Changes:
- Add disable_mcheck flag to sandbox_state
- Add mcheck_set_disabled() function to mcheck API
- Modify dlmalloc wrappers to bypass mcheck when disabled
- Add stub for when MCHECK_HEAP_PROTECTION is not enabled
- Document the new option in sandbox.rst
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a GPROF=1 build option to enable gprof profiling for sandbox. This
adds the -pg flag to both compiler and linker when GPROF=1 is set,
following the same pattern as the existing FTRACE option.
Usage:
make GPROF=1 sandbox_defconfig all
./u-boot -T -c "ut dm"
...
gprof u-boot gmon.out
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The run_test_nand() function allocates buf and gold buffers but never
frees them, leaking about 2MB per test run.
Fixes: bc8e8a4bfa ("nand: Add sandbox driver")
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The scratch buffer and glyph buffer are allocated during probe but never
freed. Add a remove callback to free these buffers when the truetype
console device is removed.
Fixes: 159af15074f2 ("video: truetype: Add a scratch buffer to use malloc() less")
Fixes: 69d2f4ab58 ("video: truetype: Use pre-allocated buffer for glyph rendering")
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The cursor save_data buffer is allocated when the cursor is enabled but
never freed. Add a pre_remove callback to free this buffer when the
vidconsole device is removed.
Fixes: aebedeac44 ("video: Provide a buffer to hold pixels behind the cursor")
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The vidconsole device name is allocated with strdup() but never marked
as allocated, so it is not freed when the device is removed. This causes
a memory leak on every video device probe/remove cycle.
Mark the name as allocated so driver model frees it on device removal,
and free it on the error path.
Fixes: 83510766c9 ("dm: video: Add a uclass for the text console")
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When IDE mode is enabled (-I), warnings are not shown because:
1. The process_result() function only shows output in verbose mode,
not IDE mode
2. When there are warnings (stderr output), the build is considered
"failed" and retried. The retry finds the object files already up
to date from the first build, so make does not recompile them and
produces no warnings. The second result (with empty stderr) then
overwrites the first, losing the warnings.
Fix this by:
- Adding IDE mode handling in process_result() to write stderr directly
- Changing the retry logic to only retry on actual failures
(return_code != 0), not on warnings
Fixes: 6a30a2666008 ("buildman: Support running from an IDE")
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a newline parameter to all output functions (info, error, warning,
notice, detail, debug, user_output, do_output) to allow suppressing the
trailing newline. This is useful for progress output where multiple
calls should appear on the same line.
Example:
tout.info('Processing...', newline=False)
tout.info('done')
Also fix typos in docstrings (msg; -> msg:).
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Call to_output() before raising CommandExc so that callers catching
the exception get string output rather than bytes. This avoids the need
for callers to handle bytes decoding themselves.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Enable the new ext4l filesystem (Linux ext4 port) for sandbox testing.
Series-to: concept
Series-cc: heinrich
Cover-letter:
ext4l: Add write support (part L)
This series adds write support to the new ext4l filesystem driver,
supporting file creation, modification, and deletion.
The main additions are:
- File write with journalling support
- File deletion (unlink)
- Directory creation (mkdir)
- Symbolic link creation (ln)
- File and directory rename
Infrastructure improvements include proper cleanup of journal references
when unmounting, safeguards to handle probe without explicit close, and
prevention of use-after-free issues with buffer_heads that have active
journal_heads.
Unlike Linux, which does not need to meticulously free every resource on
unmount (since it can do so later as needed), U-Boot must fully clean up.
The bootloader may mount and unmount filesystems multiple times during a
session, such as loading a kernel from one partition and a device tree
from another, or when running tests. Memory leaks accumulate and cause
failures, so ext4l includes extra cleanup code that the Linux ext4 driver
does not require.
The series also enables ext4l for sandbox testing and allows the use of
metadata_csum checksums which the existing ext4 driver does not support.
Some other tweaks are included:
- a fix for mcheck to avoid filling up pre-relocation malloc()
- use the correct logo for readthedocs
- add a flag to avoid using the video console in tests
- a fallback option for finding persistent data-dir
At this point ext4l is useable, but there is more work needed,
including:
- Add MAINTAINERS entry
- Add more Kconfig options for various features (to reduce code size)
- Enable for ARM and x86 targets
- Documentation
- Tests for failure cases (to check that the journal is doing its job)
END
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Series-links: 1:98
The ext4l driver supports metadata_csum checksums, unlike the old ext4
driver. Only disable metadata_csum when CONFIG_FS_EXT4L is not enabled,
allowing ext4l to use modern ext4 filesystem features.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add ext4l_rename() to rename files and directories, including moves
across directories. This uses the Linux ext4_rename() function.
Also fix the symlink test to verify reading through symlinks works
correctly, since ext4l_resolve_path follows symlinks (stat behavior).
Add Python test wrappers for mkdir, ln, and rename tests.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
The ext4l_ln() function returned -EEXIST when creating a symlink where
a file already exists. This differs from the old ext4 implementation
which deletes any existing file before creating the symlink (like ln -sf
behaviour).
Update ext4l_ln() to match this behaviour by calling __ext4_unlink() to
remove any existing non-directory file before creating the symlink.
Directories cannot be replaced with symlinks and return -EISDIR.
This allows test_symlink3 to pass.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add ext4l_ln() to create symbolic links. This uses the Linux ext4_symlink()
function which supports both fast symlinks (stored in inode) and regular
symlinks (stored in data blocks).
Fix the fscrypt_prepare_symlink() stub to properly init the disk_link
structure with the symlink target, which is required for symlink creation
to work correctly.
Add some notes about U-Boot's argument ordering with symlinks.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Implement ext4l_mkdir() to create directories on ext4 filesystems.
The function parses the path to extract the parent directory and
basename, resolves the parent inode, checks for existing entries,
and calls the Linux ext4_mkdir() function to create the directory.
Hook ext4l_mkdir into the filesystem layer via the .mkdir callback
in fs_legacy.c, enabling the standard 'mkdir' command to work with
ext4l filesystems.
Add a unit test that verifies directory creation, duplicate detection
(-EEXIST), nested directory creation, and error handling for
non-existent parent directories (-ENOENT).
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
