From 2b6e3394fb450eedc5ab46fc3ddd4361e0dd2b53 Mon Sep 17 00:00:00 2001 From: Simon Glass Date: Sat, 13 Dec 2025 14:25:23 -0700 Subject: [PATCH] test: Add documentation for test parameters Add a description of how test parameters work. This helps to make it easier to write C tests which need setup to be done in Python. Series-to: concept Series-cc: heinrich Cover-letter: test: Add support for passing arguments to C unit tests This series adds infrastructure for passing runtime arguments from Python tests to C unit tests. This makes it easier to support a hybrid testing approach where Python handles complex setup (filesystem images, environment configuration) while C handles the actual test logic with better debuggability. A few other things are included to make this work: - A fix for linker list alignment that was causing garbage values like "Running -858993444 bloblist tests" due to GCC's magic-number division optimization failing when padding breaks exact multiples - A fix fix for serial output with sandbox, since it sometimes misses output at the end when running tests with gnome terminal - Improvements to the linker-list script to detect padding and pointer-arithmetic bugs - A new UNIT_TEST_ARGS() macro for declaring tests with typed arguments, along with argument parsing in the ut command (name=value format) - Argument-accessor macros ut_str(), ut_int(), and ut_bool() with type-checking and bounds validation - A private buffer (uts->priv) for test-local temporary data, which makes it a little easier to write shorter tests - Tests for the argument feature (test_args) covering type checking, bounds checking, and argument-parsing failures As an example, the basic filesystem tests are converted from pure Python to C with Python wrappers. Some improved printf documentation and support for Linux's %pV format are provided. The slight increase in size causes qemu-riscv64_spl to fail, so this series also includes a patch to increase the SPL-malloc() space. END Co-developed-by: Claude Signed-off-by: Simon Glass --- doc/develop/tests_writing.rst | 41 +++++++++++++++++++++++++++++++++++ doc/usage/cmd/ut.rst | 11 ++++++++-- 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst index 6842533e2c6..d322cf60e26 100644 --- a/doc/develop/tests_writing.rst +++ b/doc/develop/tests_writing.rst @@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the Python test. +Passing arguments to C tests +---------------------------- + +Sometimes a C test needs parameters from Python, such as filenames or expected +values that are generated at runtime. The test-argument feature allows this. + +Use the `UNIT_TEST_ARGS` macro to declare a test with arguments:: + + static int my_test_norun(struct unit_test_state *uts) + { + const char *filename = ut_str(0); + int count = ut_int(1); + + /* test code using filename and count */ + + return 0; + } + UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite, + { "filename", UT_ARG_STR }, + { "count", UT_ARG_INT }); + +Each argument definition specifies a name and type: + +- `UT_ARG_STR` - string argument, accessed via `ut_str(n)` +- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)` +- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)` + (use `1` for true, any other value for false) + +Arguments are passed on the command line in `name=value` format:: + + ut -f my_suite my_test_norun filename=/path/to/file count=42 + +From Python, you can call the test like this:: + + cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}' + ubman.run_command(cmd) + +This approach combines Python's flexibility for setup (creating files, +generating values) with C's speed and debuggability for the actual test logic. + + How slow are Python tests? -------------------------- diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index d8c3cbf496c..a26ee6ad7de 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-r] [-f] [-I:] [ | all []] [...] + ut [-r] [-f] [-R] [-I:] [ | all []] [...] ut [-s] info Description @@ -37,10 +37,17 @@ test causes another test to fail. If the one test fails, testing stops immediately. +-R + Preserve console recording on test failure. Normally when a test fails, + console recording is disabled so error messages go directly to output. + This flag keeps recording enabled, which is useful when testing the test + framework itself. + args Optional arguments to pass to the test, in `name=value` format. These are used by tests declared with `UNIT_TEST_ARGS()` which define expected - argument names and types. + argument names and types. See :ref:`develop/tests_writing:passing arguments + to c tests` for details. Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it includes a near-complete set of emulators, no code-size limits, many CONFIG