test: Add documentation for test parameters
Add a description of how test parameters work. This helps to make it easier to write C tests which need setup to be done in Python. Series-to: concept Series-cc: heinrich Cover-letter: test: Add support for passing arguments to C unit tests This series adds infrastructure for passing runtime arguments from Python tests to C unit tests. This makes it easier to support a hybrid testing approach where Python handles complex setup (filesystem images, environment configuration) while C handles the actual test logic with better debuggability. A few other things are included to make this work: - A fix for linker list alignment that was causing garbage values like "Running -858993444 bloblist tests" due to GCC's magic-number division optimization failing when padding breaks exact multiples - A fix fix for serial output with sandbox, since it sometimes misses output at the end when running tests with gnome terminal - Improvements to the linker-list script to detect padding and pointer-arithmetic bugs - A new UNIT_TEST_ARGS() macro for declaring tests with typed arguments, along with argument parsing in the ut command (name=value format) - Argument-accessor macros ut_str(), ut_int(), and ut_bool() with type-checking and bounds validation - A private buffer (uts->priv) for test-local temporary data, which makes it a little easier to write shorter tests - Tests for the argument feature (test_args) covering type checking, bounds checking, and argument-parsing failures As an example, the basic filesystem tests are converted from pure Python to C with Python wrappers. Some improved printf documentation and support for Linux's %pV format are provided. The slight increase in size causes qemu-riscv64_spl to fail, so this series also includes a patch to increase the SPL-malloc() space. END Co-developed-by: Claude <noreply@anthropic.com> Signed-off-by: Simon Glass <simon.glass@canonical.com>
This commit is contained in:
@@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the
|
|||||||
Python test.
|
Python test.
|
||||||
|
|
||||||
|
|
||||||
|
Passing arguments to C tests
|
||||||
|
----------------------------
|
||||||
|
|
||||||
|
Sometimes a C test needs parameters from Python, such as filenames or expected
|
||||||
|
values that are generated at runtime. The test-argument feature allows this.
|
||||||
|
|
||||||
|
Use the `UNIT_TEST_ARGS` macro to declare a test with arguments::
|
||||||
|
|
||||||
|
static int my_test_norun(struct unit_test_state *uts)
|
||||||
|
{
|
||||||
|
const char *filename = ut_str(0);
|
||||||
|
int count = ut_int(1);
|
||||||
|
|
||||||
|
/* test code using filename and count */
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
|
||||||
|
{ "filename", UT_ARG_STR },
|
||||||
|
{ "count", UT_ARG_INT });
|
||||||
|
|
||||||
|
Each argument definition specifies a name and type:
|
||||||
|
|
||||||
|
- `UT_ARG_STR` - string argument, accessed via `ut_str(n)`
|
||||||
|
- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)`
|
||||||
|
- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)`
|
||||||
|
(use `1` for true, any other value for false)
|
||||||
|
|
||||||
|
Arguments are passed on the command line in `name=value` format::
|
||||||
|
|
||||||
|
ut -f my_suite my_test_norun filename=/path/to/file count=42
|
||||||
|
|
||||||
|
From Python, you can call the test like this::
|
||||||
|
|
||||||
|
cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}'
|
||||||
|
ubman.run_command(cmd)
|
||||||
|
|
||||||
|
This approach combines Python's flexibility for setup (creating files,
|
||||||
|
generating values) with C's speed and debuggability for the actual test logic.
|
||||||
|
|
||||||
|
|
||||||
How slow are Python tests?
|
How slow are Python tests?
|
||||||
--------------------------
|
--------------------------
|
||||||
|
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ Synopsis
|
|||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
|
ut [-r<runs>] [-f] [-R] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
|
||||||
ut [-s] info
|
ut [-s] info
|
||||||
|
|
||||||
Description
|
Description
|
||||||
@@ -37,10 +37,17 @@ test
|
|||||||
causes another test to fail. If the one test fails, testing stops
|
causes another test to fail. If the one test fails, testing stops
|
||||||
immediately.
|
immediately.
|
||||||
|
|
||||||
|
-R
|
||||||
|
Preserve console recording on test failure. Normally when a test fails,
|
||||||
|
console recording is disabled so error messages go directly to output.
|
||||||
|
This flag keeps recording enabled, which is useful when testing the test
|
||||||
|
framework itself.
|
||||||
|
|
||||||
args
|
args
|
||||||
Optional arguments to pass to the test, in `name=value` format. These are
|
Optional arguments to pass to the test, in `name=value` format. These are
|
||||||
used by tests declared with `UNIT_TEST_ARGS()` which define expected
|
used by tests declared with `UNIT_TEST_ARGS()` which define expected
|
||||||
argument names and types.
|
argument names and types. See :ref:`develop/tests_writing:passing arguments
|
||||||
|
to c tests` for details.
|
||||||
|
|
||||||
Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it
|
Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it
|
||||||
includes a near-complete set of emulators, no code-size limits, many CONFIG
|
includes a near-complete set of emulators, no code-size limits, many CONFIG
|
||||||
|
|||||||
Reference in New Issue
Block a user