malloc: Skip backtrace when stack is corrupted

When the stack is corrupted (e.g., by the stack protector test),
collecting a backtrace during malloc causes a crash because the
backtrace code walks the invalid stack frames.

Update __stack_chk_fail() to set the flag before calling panic()

Also update stackprot_test() to set the flag before intentionally
corrupting the stack. This is needed because of the printf() in the
test: on sandbox printf() results in truetype allocations due to the
console output.

These fixes allow the stack protector test to pass with mcheck enabled.

Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
This commit is contained in:
Simon Glass
2025-12-08 09:12:12 -07:00
parent 754a755e4a
commit a527909208
2 changed files with 13 additions and 0 deletions

View File

@@ -4,6 +4,7 @@
*/
#include <asm/global_data.h>
#include <malloc.h>
DECLARE_GLOBAL_DATA_PTR;
@@ -13,6 +14,11 @@ void __stack_chk_fail(void)
{
void *ra;
/*
* When the stack is corrupted, backtrace collection will crash.
* Skip it before calling panic().
*/
malloc_backtrace_skip(true);
ra = __builtin_extract_return_addr(__builtin_return_address(0));
panic("Stack smashing detected in function:\n%p relocated from %p",
ra, ra - gd->reloc_off);