Add C implementations of filesystem tests that can be called via
the 'ut fs' command. These tests use UTF_MANUAL flag since they require
external setup, i.e. creation of filesystem images.
This covers the existing TestFsBasic tests.
The tests use typed arguments (fs_type, fs_image, md5 values) passed
via the command line.
Add a few helpers to make the code easier to read.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Update the 'ut' command to have a -R command option to prevent ut_fail()
and ut_failf() from clearing GD_FLG_RECORD. This is useful when testing
the test framework itself, where error messages need to be captured.
Refactor ut_fail() and ut_failf() to call ut_unsilence_console() instead
of duplicating the flag-clearing logic.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a priv[] buffer to struct unit_test_state that tests can use for
their own data. This avoids the need to allocate memory or use global
variables for test-specific state.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add ut_get_str(), ut_get_int(), and ut_get_bool() functions with
corresponding ut_str(), ut_int(), and ut_bool() macros for accessing
test arguments with type checking.
These functions check that the argument index is within bounds and the
type matches what was requested.
The first failure for a test is reported via ut_failf() which should
make it fairly easy to debug the test.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add support for passing key=value arguments to unit tests. The test
framework parses arguments based on definitions provided by each test
and makes them available via uts->args[]
For now the 'ut' command does not support this new feature.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a UNIT_TEST_ARGS() macro to declare and register a test with inline
argument definitions. The variadic arguments use struct ut_arg_def and
a NULL terminator is added automatically to the list.
Example usage:
static int my_test(struct unit_test_state *uts) { ... }
UNIT_TEST_ARGS(my_test, UTF_CONSOLE, my_suite,
{ "path", UT_ARG_STR },
{ "count", UT_ARG_INT, UT_ARGF_OPTIONAL, { .vint = 10 } });
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add types for declaring and storing unit test arguments:
- enum ut_arg_type: INT, BOOL, STR types
- enum ut_arg_flags: OPTIONAL flag for non-required args
- struct ut_arg_def: declares expected args with defaults
- struct ut_arg: holds parsed argument values
This prepares for passing key=value arguments to tests via the 'ut'
command instead of needing to use environment variables.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a new ut_assert_nextline_regex() macro and ut_check_console_line_regex()
helper to check console output against a regex pattern. This is useful when
the exact output varies (e.g., file paths or line numbers in error messages).
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a new assertion macro ut_asserteq_regex() that checks if a string
matches a regular expression pattern using the SLRE library.
This is useful for tests where exact string-matching is difficult, such
as when output contains line numbers or other variable content.
Use a helper function ut_check_regex() to avoid including slre.h in the
header.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Some tests cannot run when the console is silent.
An example is an expo test which checks text entry into a textline
object. The console must be enabled so that the characters actually
reach the putc_xy() in console_truetype, since in
scene_textline_send_key(), the lineedit restores the vidconsole state,
outputs the character and then saves the state again. If the character
is never output, then the state won't be updated and the lineedit will
be inconsistent.
Rather than having individual tests handle this manually, add an
explicit flag, in the hope that this quirk does not trip anyone else up.
Put the flag next to the existing UTF_CONSOLE flag, since they are
related.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
These buffers can be exceeded when checking the FIT output with an RSA
key. Increase the limit to 1K.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Update ut_asserteq_str() and ut_asserteq_strn() to check for NULL. This
allows tests to avoid doing this.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
At present boxes are not supported in the expo_build format. Also, it is
now possible to draw filled boxes, a recently added feature to the video
API.
Expand the box feature slightly to resolve these two items.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Normally the first failed assertion causes the test to stop. Provide a
flag to allow the test to continue, as a means to discover other
problems later in the test.
Note that the utility of this depends on the test, since some tests will
be broken if any step fails. But this can be useful when there are lots
of test adjustments to make.
Signed-off-by: Simon Glass <sjg@chromium.org>
When OF_LIVE is enabled, sandbox runs tests with that and with flattree.
When debugging tests, this is often just a distraction.
Provide a -F option to skip the flattree tests in this case.
Signed-off-by: Simon Glass <sjg@chromium.org>
We want to check the display contents in expo tests, so move the two
needed functions to a new header file.
Rename them to have a video_ prefix.
Signed-off-by: Simon Glass <sjg@chromium.org>
Now that everything is using the new test-suite features, drop support
for running commands.
Fix a missing closing-bracket while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than having an init function and then running the tests, create a
test-init function to do it. This will allow us to get rid of the
command function.
Fix the comment abotu 'environment' while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than having an init function and then running the tests, create a
test-init function to do it. This will allow us to get rid of the
command function.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some suites need things to be set up before they can run. Add a way to
declare an init function using the UNIT_TEST_INIT() macro. The init
function is just like any other test, but is always placed first so that
it runs before all the other test functions in the suite.
Add an uninit function as well, to clean up after the test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Show the average duration of a test, so we can keep track of how it is
trending. Report the suite with the longest average test to encourage
people to improve it.
Add a function to update the stats based on the results from a single
suite and another to show the summary information.
Make this optional, since sandbox's SPL tests do not have a timer driver
and people may want to print results without times.
Signed-off-by: Simon Glass <sjg@chromium.org>
Show the time taken by each test suite with 'ut all' and the total time
for all suites.
Take care to remove any sandbox time-offset from the values.
Fix the comment-format on timer_test_add_offset() while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a function to show the stats, so we can decide when to print it.
This slightly adjusts the output, so that any 'test not found' message
appears on its own line after all other output.
The 'failures' message now appears in lower case so update pytest
accordingly.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is useful information and is not always the same as the 'count' arg
to ut_run_list() so add it as a separate stat.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use a struct to hold the stats, since we also want to have the same
stats for all runs as we have for each suite.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update this function to access a unit-test state, so that the caller can
collect results from running multiple suites.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use the new suite-runner to run these tests instead.
It is not clear that these actually work, since they are not enabled on
sandbox for some reason.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use the new suite-runner to run these tests instead.
It is not clear that these actually work, since they are not enabled on
sandbox for some reason.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
