4363 Commits

Author SHA1 Message Date
Simon Glass
5694ce93bd patman: Add a test for collecting a PATCH prefix
Add one more test for completeness, since this occurs commonly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-24 05:26:46 +01:00
Simon Glass
fc91a400f4 patman: Use python3 with /usr/bin/env
If python2 is available then we must specify python3 here. Update the
test to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-24 05:26:46 +01:00
Simon Glass
b5f435c7a3 u_boot_pylib: Add more functions to gitutil
Add functions for checking a branch, showing a commit, etc. to support
the new functionality.

Git version 2.34.1 ignores --stat if --quiet is given, so adjust the
args so that this performs as expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-24 05:26:46 +01:00
Simon Glass
1c35dcfcef patman: Deal with git safe-directory warning
When running tests where the .git directory is not owned by the current
user, various warnings are produced and the tests fail. This happens in
CI.

For patman itself, modify the gitutil.get_top_level() function to return
None in this case. Ensure that the warning is not shown, since it creates
about 1000 lines of output.

For checkpatch, the same warning is produced even though --no-tree is
given. Suppress that as well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-24 05:26:46 +01:00
Simon Glass
7db8c14065 u_boot_pylib: Correct pylint warnings in gitutil
Correct various pylint warnings in this file.

The remaining ones are three functions with too many arguments (R0913
and R0918) and use of global (W0603).

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-24 05:26:46 +01:00
Simon Glass
20131bff66 patman: Support collecting all lines in the commit message
In some cases we want to collect all lines in the commit message so that
the commit can be recreated with the same message as before, or perhaps
with light filtering.

Add support for this.

Series-to: u-boot
Cover-letter:
patman: Minor improvements to prepare for series handling
This series includes a number of internal improvements to patman:
- Tidy-up of parsing
- Adjust how tests create the git tree
- Support for creating patches in a different git directory
- Faster determination of the upstream branch
- Ability to collect the body of a commit message as a series of lines
END

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
b7c0f7b0e6 patman: Tidy up function comments in the series module
This module is missing some comments, so add them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
8158f46a25 patman: Move -H out of the send command
This is the help for the whole of patman, so move it to the start of the
control function, rather than being inside 'patman send'.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
5734b2b7bc patman: Move arguments for sent into the correct parser
Most of the arguments for the main parser are actually arguments for the
'send' parser. Move them there, in a separate function.

Fix a pylint warning for -D and the imports while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
bf692cc795 patman: Split subparsers into their own functions
Simplify the main parser by moving subparser code into separate
functions. Fix a few pylint warnings while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
4ceb95daab patman: Add tests for help
Add a few tests to make sure that help is provided when requested.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
5c5eb063bd patman: Split parser creation from parsing
Tests may want to parse their own arguments. Refactor the parser code to
support this and allow settings to receive arguments as well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
6ade0733e9 patman: Correct some pylint and asyncio issues
Correct some pylint warnings in this file. Make use of the existing
asyncio event loop, instead of creating a new one, since this also
destroys it afterwards, making it unavailable for tests which want to
share an event loop. Use tools.write_file() to avoid a warning about
encoding.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
797cc447a4 patman: Use git to set up the test config
At present the branch information is dropped when writing the
configuration. It is easier to get git to set up the config anyway, so
update the test to do this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
329636499c patman: Clean up creation of the git tree
The test starts with the HEAD pointing to the wrong place, so that the
created files appear to be deleted. Fix this by resetting the tree
before tests start. Add a check that the tree is clean.

Update pygit2 so that the enums are available.
2025-05-23 17:10:12 +01:00
Simon Glass
41977a9541 patman: Use variables for series ID and title
Rather than hard-coding these values in the sample patches, use
variables so that we can refer to these in tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
23028aa1fe patman: Use .git as the git directory
In tests, the 'git' directory is a subdirectory of the temporary
directory. Rename it to '.git' so that git will automatically find it
when git operations are done in the temporary directory. Set up the
config before the first git operation, so that this works correctly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
766e1a2d0c patman: Use itr instead of lines for iterator
The variables 'lines' is used to hold a list of output lines within a
test, but also to hold an iterator through those lines. Use 'itr' for
the latter, to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
f4c1ebe080 patman: Correct use of str in code
Since str() is a reserved function we should not use it as a variable.
Fix this in the send module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
488faefc26 patman: Allow setting a git directory when sending
Support specifying the git-directory when creating and sending patches.
This will allow better testing of this functionality, since we can use a
test directory.

For count_commits_to_branch() support an end commit while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
8404c0f72c patman: Allow setting the current directory when sending
Plumb a current-working-directory (cwd) through from send all the way to
the command gitutil libraries. This will allow better testing of this
functionality, since we can use a test directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
a94d138f0f patman: Add all files to __init__.py
Some files are missing from the __all__ list, so add then. Reformat the
list to use more of the width of each line.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
9229e1cdb4 u_boot_pylib: Speed up determining the upstream branch
Use --decorate to quickly detect the upstream branch, since this is much
faster than using 'git name-rev' on every possible commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
ba42eb41b6 u_boot_pylib: Provide directories to gitutil functions
For testing it is useful to be able to set the current directory used
for git operations, as well as the git-repo directory. Update some of
the functions to support this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:10:12 +01:00
Simon Glass
dfd60356f8 u_boot_pylib: Tidy up quoting of cc and to
The current approach to calling 'git send-email' puts double quotes
around each email address to ensure that it will pass the shell
correctly. This is a bit cumbersome and requires using a shell to sort
it all out.

Drop the quotes and use command.run() instead, to simplify things. This
will also make it possible to (later) set the current directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-23 17:06:29 +01:00
Tom Rini
a2ef6911d0 Docker, CI: Add vexpress_fvp / vexpress_fvp_bloblist support
This adds the vexpress_fvp and vexpress_fvp_bloblist platforms to the
list of platforms we test via emulator in CI. In order to do this we
need to first have our container runtime have TF-A builds for the
vexpress_fvp platform, both with and without transfer list support as
well as installing "telnet" so that we can access console. In the CI
files we check for the existence of /opt/tf-a/${TEST_PY_BD} and if
found, copy bl1.bin and fip.bin to /tmp and set the variables so that we
can later run FVP to run.

Note that we currently disable the hostfs (semihosting) tests as they
trigger a bug in FVP. This has been reported upstream, and can be
enabled when fixed.

Reviewed-by: Harrison Mutai <harrison.mutai@arm.com>
Acked-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2025-05-16 07:36:53 +00:00
Simon Glass
8ca250c542 u_boot_pylib: Support a fatal level in tout
It is convenient to be able to print a message and exit. Add a new
'fatal' level to support this.

Update some assumptions about the level, so that the tools continue to
work as now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-15 06:35:37 +02:00
Simon Glass
30035868bd patman: Update Series.ShowActions() to pass alias
Instead of using settings.alias pass this value in. This allows tests to
work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:19:20 -06:00
Simon Glass
081889ba77 patman: Pass aliases to Series.MakeCcFile()
Rather than accessing settings directly, pass the aliases in, so that
we can do the same from tests. With further work this will allow the
tests to work without using settings.alias

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:18:30 -06:00
Simon Glass
c6a8025072 patman: Pass the alias dict into gitutil.email_patches()
Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:17:20 -06:00
Simon Glass
8f2279a567 patman: Pass the alias dict into gitutil.build_email_list()
Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:15:06 -06:00
Simon Glass
57e0dbd970 patman: Untangle settings from gitutil
The gitutil module is supposed to be independent from patman but one
piece was missed in the series which separated them.

Move the settings setup out of gitutil

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:15:05 -06:00
Simon Glass
10ecf6ce9c patman: Drop the old status code
Now that we have the new asyncio implementation, drop the unused
ThreadPoolExecutor code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
4b8d7809cb patman: Use a pager when outputting status
The pager makes it easier to review large number of comments. Add it
around the output.

Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
1d8bb45a2a patman: Provide an option to run in single-threaded mode
Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
761d34ca46 patman: Switch over to asyncio
Now that all the pieces are in place, switch over from using an executor
to using asyncio.

While we are here, import defaultdict to avoid needing to specify its
module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
18db7ee440 patman: Use defaultdict directly
Import defaultdict to avoid needing to specify its module in multiple
places. Fix an extra blank line while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
4967b8e7c8 patman: Add more information to Patch
The cover letter has some information on each patch, so allow this to be
stored in the Patch object.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
fec781a613 patman: Show patches in yellow
When comments are shown below patches it can be hard to see the patch
subject. Use yellow instead of blue, since it stands out better.

Pass the colour object into show_responses() rather than creating a new
one, since that is tidier.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
ce33107be8 patman: Simplify test_find_new_responses()
This test uses the find_new_responses() function which combines
accessing patchwork and processing the results.

Since the test is aimed at testing processing, adjust it to call
process_reviews() instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
f64e147d83 patman: Adjust how the fake request() function is provided
Instead of passing the URL and function to each call, put the fake
into the Patchwork object instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
cb61ab9daa patman: Split out the processing of find_new_responses()
This function does patchwork calls and then processes the results. Move
the processing out so we can (later) test it separately.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
805292aeb2 patman: Split up check_and_show_status()
This function has three phases:

- collecting things from patchwork
- doing some processing
- showing the results to the user / creating a branch

Refactor into two functions so we can eventually have the patchwork part
fully separated out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
bde04a28aa patman: Rename check_patchwork_status()
This function actually shows the status and does some other things.

Rename it to better reflect its purpose.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
ffb56ff956 patman: Add reading of series and patch status
Expand the patchwork module so that it can match the current
requirements of the 'patman status' command, i.e. reading the state of a
series and the patches associated with it.

Since the format of each patchwork response is a little tricky to
understand, add examples in comments at the top of each function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
5df55c4a06 patman: Move Patch and Review to patchwork module
These relate to information obtained from the patchwork server, so move
their definition into the new patchwork module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
ca29eb5390 patman: Create a module for handling patchwork
At present the patchwork implementation is very simple, just consisting
of a function which calls the REST API.

We want to create a fake patchwork for use in tests. So start a new
module to encapsulate communication with the patchwork server.

Use asyncio since it is easier to handle lots of concurrent requests
from different parts of the code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:03 -06:00
Simon Glass
2327811fc0 CI: Add a dependency file for patman
Now that patman has an unusual dependency, add a requirements.txt file
and use it in CI

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:09:00 -06:00
Simon Glass
52938326df patman: Move code for sending into its own module
The control module includes much of the implementation of patman's
'send' feature. As a first step to separating this out, move the sending
code into its own file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:03:34 -06:00
Simon Glass
4528ccc9c3 patman: Support extra test features
Provide support for the -X flag, which preserves the working directory
used by tests. Also support -N which shows captured output for tests.

Finally, allow selection of a particular test to run.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-01 20:03:34 -06:00