test: Add documentation for test parameters
Add a description of how test parameters work. This helps to make it easier to write C tests which need setup to be done in Python. Series-to: concept Series-cc: heinrich Cover-letter: test: Add support for passing arguments to C unit tests This series adds infrastructure for passing runtime arguments from Python tests to C unit tests. This makes it easier to support a hybrid testing approach where Python handles complex setup (filesystem images, environment configuration) while C handles the actual test logic with better debuggability. A few other things are included to make this work: - A fix for linker list alignment that was causing garbage values like "Running -858993444 bloblist tests" due to GCC's magic-number division optimization failing when padding breaks exact multiples - A fix fix for serial output with sandbox, since it sometimes misses output at the end when running tests with gnome terminal - Improvements to the linker-list script to detect padding and pointer-arithmetic bugs - A new UNIT_TEST_ARGS() macro for declaring tests with typed arguments, along with argument parsing in the ut command (name=value format) - Argument-accessor macros ut_str(), ut_int(), and ut_bool() with type-checking and bounds validation - A private buffer (uts->priv) for test-local temporary data, which makes it a little easier to write shorter tests - Tests for the argument feature (test_args) covering type checking, bounds checking, and argument-parsing failures As an example, the basic filesystem tests are converted from pure Python to C with Python wrappers. Some improved printf documentation and support for Linux's %pV format are provided. The slight increase in size causes qemu-riscv64_spl to fail, so this series also includes a patch to increase the SPL-malloc() space. END Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com>
This commit is contained in:
@@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the
|
||||
Python test.
|
||||
|
||||
|
||||
Passing arguments to C tests
|
||||
----------------------------
|
||||
|
||||
Sometimes a C test needs parameters from Python, such as filenames or expected
|
||||
values that are generated at runtime. The test-argument feature allows this.
|
||||
|
||||
Use the `UNIT_TEST_ARGS` macro to declare a test with arguments::
|
||||
|
||||
static int my_test_norun(struct unit_test_state *uts)
|
||||
{
|
||||
const char *filename = ut_str(0);
|
||||
int count = ut_int(1);
|
||||
|
||||
/* test code using filename and count */
|
||||
|
||||
return 0;
|
||||
}
|
||||
UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
|
||||
{ "filename", UT_ARG_STR },
|
||||
{ "count", UT_ARG_INT });
|
||||
|
||||
Each argument definition specifies a name and type:
|
||||
|
||||
- `UT_ARG_STR` - string argument, accessed via `ut_str(n)`
|
||||
- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)`
|
||||
- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)`
|
||||
(use `1` for true, any other value for false)
|
||||
|
||||
Arguments are passed on the command line in `name=value` format::
|
||||
|
||||
ut -f my_suite my_test_norun filename=/path/to/file count=42
|
||||
|
||||
From Python, you can call the test like this::
|
||||
|
||||
cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}'
|
||||
ubman.run_command(cmd)
|
||||
|
||||
This approach combines Python's flexibility for setup (creating files,
|
||||
generating values) with C's speed and debuggability for the actual test logic.
|
||||
|
||||
|
||||
How slow are Python tests?
|
||||
--------------------------
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ Synopsis
|
||||
|
||||
::
|
||||
|
||||
ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
|
||||
ut [-r<runs>] [-f] [-R] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
|
||||
ut [-s] info
|
||||
|
||||
Description
|
||||
@@ -37,10 +37,17 @@ test
|
||||
causes another test to fail. If the one test fails, testing stops
|
||||
immediately.
|
||||
|
||||
-R
|
||||
Preserve console recording on test failure. Normally when a test fails,
|
||||
console recording is disabled so error messages go directly to output.
|
||||
This flag keeps recording enabled, which is useful when testing the test
|
||||
framework itself.
|
||||
|
||||
args
|
||||
Optional arguments to pass to the test, in `name=value` format. These are
|
||||
used by tests declared with `UNIT_TEST_ARGS()` which define expected
|
||||
argument names and types.
|
||||
argument names and types. See :ref:`develop/tests_writing:passing arguments
|
||||
to c tests` for details.
|
||||
|
||||
Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it
|
||||
includes a near-complete set of emulators, no code-size limits, many CONFIG
|
||||
|
||||
Reference in New Issue
Block a user