test: Add documentation for test parameters

Add a description of how test parameters work. This helps to make it
easier to write C tests which need setup to be done in Python.

Series-to: concept
Series-cc: heinrich
Cover-letter:
test: Add support for passing arguments to C unit tests
This series adds infrastructure for passing runtime arguments from Python
tests to C unit tests. This makes it easier to support a hybrid testing
approach where Python handles complex setup (filesystem images,
environment configuration) while C handles the actual test logic with
better debuggability.

A few other things are included to make this work:

- A fix for linker list alignment that was causing garbage values like
  "Running -858993444 bloblist tests" due to GCC's magic-number division
  optimization failing when padding breaks exact multiples

- A fix fix for serial output with sandbox, since it sometimes misses
  output at the end when running tests with gnome terminal

- Improvements to the linker-list script to detect padding and
  pointer-arithmetic bugs

- A new UNIT_TEST_ARGS() macro for declaring tests with typed arguments,
  along with argument parsing in the ut command (name=value format)

- Argument-accessor macros ut_str(), ut_int(), and ut_bool() with
  type-checking and bounds validation

- A private buffer (uts->priv) for test-local temporary data, which
  makes it a little easier to write shorter tests

- Tests for the argument feature (test_args) covering type checking,
  bounds checking, and argument-parsing failures

As an example, the basic filesystem tests are converted from pure Python
to C with Python wrappers.

Some improved printf documentation and support for Linux's %pV format
are provided.

The slight increase in size causes qemu-riscv64_spl to fail, so this
series also includes a patch to increase the SPL-malloc() space.
END

Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
This commit is contained in:
Simon Glass
2025-12-13 14:25:23 -07:00
parent 32c67c43fb
commit 2b6e3394fb
2 changed files with 50 additions and 2 deletions

View File

@@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the
Python test.
Passing arguments to C tests
----------------------------
Sometimes a C test needs parameters from Python, such as filenames or expected
values that are generated at runtime. The test-argument feature allows this.
Use the `UNIT_TEST_ARGS` macro to declare a test with arguments::
static int my_test_norun(struct unit_test_state *uts)
{
const char *filename = ut_str(0);
int count = ut_int(1);
/* test code using filename and count */
return 0;
}
UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
{ "filename", UT_ARG_STR },
{ "count", UT_ARG_INT });
Each argument definition specifies a name and type:
- `UT_ARG_STR` - string argument, accessed via `ut_str(n)`
- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)`
- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)`
(use `1` for true, any other value for false)
Arguments are passed on the command line in `name=value` format::
ut -f my_suite my_test_norun filename=/path/to/file count=42
From Python, you can call the test like this::
cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}'
ubman.run_command(cmd)
This approach combines Python's flexibility for setup (creating files,
generating values) with C's speed and debuggability for the actual test logic.
How slow are Python tests?
--------------------------

View File

@@ -11,7 +11,7 @@ Synopsis
::
ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
ut [-r<runs>] [-f] [-R] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
ut [-s] info
Description
@@ -37,10 +37,17 @@ test
causes another test to fail. If the one test fails, testing stops
immediately.
-R
Preserve console recording on test failure. Normally when a test fails,
console recording is disabled so error messages go directly to output.
This flag keeps recording enabled, which is useful when testing the test
framework itself.
args
Optional arguments to pass to the test, in `name=value` format. These are
used by tests declared with `UNIT_TEST_ARGS()` which define expected
argument names and types.
argument names and types. See :ref:`develop/tests_writing:passing arguments
to c tests` for details.
Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it
includes a near-complete set of emulators, no code-size limits, many CONFIG