Add a newline parameter to all output functions (info, error, warning,
notice, detail, debug, user_output, do_output) to allow suppressing the
trailing newline. This is useful for progress output where multiple
calls should appear on the same line.
Example:
tout.info('Processing...', newline=False)
tout.info('done')
Also fix typos in docstrings (msg; -> msg:).
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Call to_output() before raising CommandExc so that callers catching
the exception get string output rather than bytes. This avoids the need
for callers to handle bytes decoding themselves.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add support for running pickman tests with code coverage checking,
similar to binman. Use test_util.run_test_suites() for running tests
and test_util.run_test_coverage() for coverage checking.
Options added to the 'test' command:
-P: Number of processes for parallel test execution
-T: Run with coverage checking
-v: Verbosity level (0-4)
The coverage check allows failures for modules that require external
services (agent.py, gitlab_api.py, control.py, database.py) since
these cannot easily achieve 100% coverage in unit tests.
Also update test_util.py to recognize 'pickman' as using the 'test'
subcommand like binman and patman.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Refactor tout.py to use terminal.tprint() instead of direct print()
calls. This provides better control over output formatting and
supports the new stderr parameter. It also reduces code duplication.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add optional stderr parameter to tprint() to allow printing to stderr
instead of stdout.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add functions for checking a branch, showing a commit, etc. to support
the new functionality.
Git version 2.34.1 ignores --stat if --quiet is given, so adjust the
args so that this performs as expected.
Signed-off-by: Simon Glass <sjg@chromium.org>
When running tests where the .git directory is not owned by the current
user, various warnings are produced and the tests fail. This happens in
CI.
For patman itself, modify the gitutil.get_top_level() function to return
None in this case. Ensure that the warning is not shown, since it creates
about 1000 lines of output.
For checkpatch, the same warning is produced even though --no-tree is
given. Suppress that as well.
Signed-off-by: Simon Glass <sjg@chromium.org>
Correct various pylint warnings in this file.
The remaining ones are three functions with too many arguments (R0913
and R0918) and use of global (W0603).
Signed-off-by: Simon Glass <sjg@chromium.org>
The test starts with the HEAD pointing to the wrong place, so that the
created files appear to be deleted. Fix this by resetting the tree
before tests start. Add a check that the tree is clean.
Update pygit2 so that the enums are available.
Use --decorate to quickly detect the upstream branch, since this is much
faster than using 'git name-rev' on every possible commit.
Signed-off-by: Simon Glass <sjg@chromium.org>
For testing it is useful to be able to set the current directory used
for git operations, as well as the git-repo directory. Update some of
the functions to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
The current approach to calling 'git send-email' puts double quotes
around each email address to ensure that it will pass the shell
correctly. This is a bit cumbersome and requires using a shell to sort
it all out.
Drop the quotes and use command.run() instead, to simplify things. This
will also make it possible to (later) set the current directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is convenient to be able to print a message and exit. Add a new
'fatal' level to support this.
Update some assumptions about the level, so that the tools continue to
work as now.
Signed-off-by: Simon Glass <sjg@chromium.org>
The gitutil module is supposed to be independent from patman but one
piece was missed in the series which separated them.
Move the settings setup out of gitutil
Signed-off-by: Simon Glass <sjg@chromium.org>
When running tests there is no situation in which we want ANSI output as
it makes it much harder to see what is going on in logs, tests, etc.
Provide a way to disable this.
Signed-off-by: Simon Glass <sjg@chromium.org>
There is no point in spinning up multiple processes if there is only one
test to execute. Add a check for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to have a pager when outputting a lot of text. Add support
for this in the terminal library, making use of a context manager.
Also add a function to indicate whether the output device is a terminal
or not, while we are here, to avoid duplicating this code.
Signed-off-by: Simon Glass <sjg@chromium.org>
The current implementation does not handle background colours very well:
- It outputs an incorrect code in some cases, leading to wrong colours
- Some functions lack a control for the background
Tidy this up so that background colours can be used in more places.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tests often capture output so they can check it. This means that if the
test fails it is not easy to see what the output actually was.
Add a -N flag which writes out the output after it has been captured.
This is not a perfect solution but it is simple and seems to work well
in practice.
Signed-off-by: Simon Glass <sjg@chromium.org>
This function is sometimes useful outside tests. Also it can affect how
terminal output is done, e.g. whether ANSI characters should be emitted
or not.
Move it out of the test_util package and into terminal.
Signed-off-by: Simon Glass <sjg@chromium.org>
Using some form of sandbox with Python modules is a long standing best
practice with the language. There are a number of ways to have a Python
sandbox be created. At this point in time, it seems the Python community
is moving towards using the "venv" module provided with Python rather
than a separate tool. To match that we make the following changes:
- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not
virtualenv.
Signed-off-by: Tom Rini <trini@konsulko.com>
Rather than have a requirements.txt file that's shared between multiple
python projects within U-Boot, create one for each using "pipreqs".
Signed-off-by: Tom Rini <trini@konsulko.com>
It is helpful to know which commit patches are based on, even if that
commit might not be available to readers. Add a tag for this in the
cover letter.
Also add the local-branch name since that may be useful to the writer.
Signed-off-by: Simon Glass <sjg@chromium.org>
Newer versions of the python3-coverage tool require a directory
separator before and after the directory name. Add this so that system
package are not included in the coverage report.
Signed-off-by: Simon Glass <sjg@chromium.org>
append_input_dirs() can be used to append a list of input directories to indir
global list.
Signed-off-by: Paul HENRYS <paul.henrys_ext@softathome.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add a helper to avoid needing to use a list within a list for this
simple case.
Update existing users of runpipe() to use this where possible.
Signed-off-by: Simon Glass <sjg@chromium.org>
This file has a lot of warnings. Before adding any more features, fix
those which are straightforward to resolve.
Signed-off-by: Simon Glass <sjg@chromium.org>
Throwing an Exception is not very friendly since it is the top-level
class of all exceptions. Declare a new class instead.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present run_test_coverage() assumes you want code coverage for the
entire code base. This is the normal situation, but sometimes it is
useful to see the coverage provided by just a single test. Add support
for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
When running within a Python venv we must use the 'coverage' tool (which
is within the venv) so that the venv packages are used in preference to
system packages. Otherwise the coverage tests run in a different
environment from the normal tests and may fail due to missing packages.
Handle this by detecting the venv and changing the tool name.
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman uses all available CPUs by default, so running more than one or
two concurrent processes is not normally useful.
However in some CI cases we want to be able to run several jobs at once
to save time. For example, in a lab situation we may want to run a test
on 20 boards at a time, since only the build step actually takes much
CPU.
Add an option which allows such a limit. When buildman starts up, it
waits until the number of running processes goes below the limit, then
claims a spot in the list. The list is maintained with a temporary file.
Note that the temp file is user-specific, since it is hard to create a
locked temporary file which can be accessed by any user. In most cases,
only one user is running jobs on a machine, so this should not matter.
Signed-off-by: Simon Glass <sjg@chromium.org>
A new release has been done with this version, so update it. Use the
version numbers in dependencies also.
Signed-off-by: Simon Glass <sjg@chromium.org>
The files list is incorrect and dates from a time when the script was
run from a different directory. Update it to match all the other tools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Correct a heading which is too short in the readme.
Fixes: 75554dfac2 ("patman: Add support for building a u_boot_tools...")
Signed-off-by: Simon Glass <sjg@chromium.org>
It isn't clear why we need to have two different paths for closing down
the pipe. Unify them and use the Python to avoid this warning:
subprocess.py:1127: ResourceWarning: subprocess 83531 is still running
Note that this code appears to originally have come from [1] and was
committed into the ChromeOS chromiumos/platform/crosutils repo in the
bin/cros_image_to_target.py file. The addition of the extra code path
came later, so that is chosen for the fixes tag.
[1] https://codereview.chromium.org/3391008
Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: a10fd93cbc patman: Make command methods return a CommandResult
When building from source, setuptools would complain about not finding
package via its auto-discovery mechanism. Manually specify how to
locate the files, relative to the package's directory.
* tools/u_boot_pylib/pyproject.toml: New tool.setuptools.packages.find
section.
Signed-off-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.
To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.
Signed-off-by: Simon Glass <sjg@chromium.org>
