test: Add a way to detect a test that breaks another

When running unit tests, some may have side effects which cause a
subsequent test to break. This can sometimes be seen when using 'ut dm'
or similar.

Add a new argument which allows a particular (failing) test to be run
immediately after a certain number of tests have run. This allows the
test causing the failure to be determined.

Update the documentation also.

Signed-off-by: Simon Glass <sjg@chromium.org>
This commit is contained in:
Simon Glass
2022-10-29 19:47:13 -06:00
parent 6580b61830
commit d1b4659570
6 changed files with 127 additions and 10 deletions

View File

@@ -8,10 +8,12 @@ Synopis
::
ut [-r<runs>] [-f] [<suite> [<test>]]
ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> [<test>]]
<runs> Number of times to run each test
-f Force 'manual' tests to run as well
<n> Run <one test> after <n> other tests have run
<one_test> Name of the 'one' test to run
<suite> Test suite to run, or `all`
<test> Name of single test to run
@@ -35,6 +37,13 @@ Manual tests are normally skipped by this command. Use `-f` to run them. See
See :ref:`develop/tests_writing:mixing python and c` for more information on
manual test.
When running unit tests, some may have side effects which cause a subsequent
test to break. This can sometimes be seen when using 'ut dm' or similar. To
fix this, select the 'one' test which breaks. Then tell the 'ut' command to
run this one test after a certain number of other tests have run. Using a
binary search method with `-I` you can quickly figure one which test is causing
the problem.
Generally all tests in the suite are run. To run just a single test from the
suite, provide the <test> argument.