Compare commits

..

26 Commits
pat ... pata

Author SHA1 Message Date
Simon Glass
3b7819603f patman: Drop the old status code
Now that we have the new asyncio implementation, drop the unused
ThreadPoolExecutor code.

Series-to: u-boot
Cover-letter:
patman: Move to asyncio and other updates
This series moves patman to use asyncio instead of ThreadPoolExecutor
since it makes it easier to handle requests sent from different parts
of the code.

It also includes some updates to bring patman in line with the other
major Python tools (buildman and binman).

With this series, we have a better basis to add support for syncing more
data from a patchwork server.

After this, the next step it to add a new 'patman series' subcommand, to
track and manage series workflow.
END

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:22:06 -06:00
Simon Glass
b18f977c89 patman: Use a pager when outputting status
The pager makes it easier to review large number of comments. Add it
around the output.

Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
8032c7102a patman: Provide an option to run in single-threaded mode
Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
527673b00a patman: Switch over to asyncio
Now that all the pieces are in place, switch over from using an executor
to using asyncio.

While we are here, import defaultdict to avoid needing to specify its
module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
4bd713247f patman: Use defaultdict directly
Import defaultdict to avoid needing to specify its module in multiple
places. Fix an extra blank line while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
d22c10179b patman: Add more information to Patch
The cover letter has some information on each patch, so allow this to be
stored in the Patch object.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
64aec306ea patman: Show patches in yellow
When comments are shown below patches it can be hard to see the patch
subject. Use yellow instead of blue, since it stands out better.

Pass the colour object into show_responses() rather than creating a new
one, since that is tidier.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
9f0fae8959 patman: Simplify test_find_new_responses()
This test uses the find_new_responses() function which combines
accessing patchwork and processing the results.

Since the test is aimed at testing processing, adjust it to call
process_reviews() instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
af2f0b9b14 patman: Adjust how the fake request() function is provided
Instead of passing the URL and function to each call, put the fake
into the Patchwork object instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
840c910445 patman: Split out the processing of find_new_responses()
This function does patchwork calls and then processes the results. Move
the processing out so we can (later) test it separately.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
5d29555f0e patman: Split up check_and_show_status()
This function has three phases:

- collecting things from patchwork
- doing some processing
- showing the results to the user / creating a branch

Refactor into two functions so we can eventually have the patchwork part
fully separated out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
e7cf9d4400 patman: Rename check_patchwork_status()
This function actually shows the status and does some other things.

Rename it to better reflect its purpose.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
9866a27689 patman: Add reading of series and patch status
Expand the patchwork module so that it can match the current
requirements of the 'patman status' command, i.e. reading the state of a
series and the patches associated with it.

Since the format of each patchwork response is a little tricky to
understand, add examples in comments at the top of each function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
1fcdb4cd02 patman: Move Patch and Review to patchwork module
These relate to information obtained from the patchwork server, so move
their definition into the new patchwork module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
a4f0680e54 patman: Create a module for handling patchwork
At present the patchwork implementation is very simple, just consisting
of a function which calls the REST API.

We want to create a fake patchwork for use in tests. So start a new
module to encapsulate communication with the patchwork server.

Use asyncio since it is easier to handle lots of concurrent requests
from different parts of the code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
12d4ec2650 CI: Add a dependency file for patman
Now that patman has an unusual dependency, add a requirements.txt file
and use it in CI

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
a288523e65 patman: Move code for sending into its own module
The control module includes much of the implementation of patman's
'send' feature. As a first step to separating this out, move the sending
code into its own file.
2025-04-29 07:09:55 -06:00
Simon Glass
19bee3dbad patman: Support extra test features
Provide support for the -X flag, which preserves the working directory
used by tests. Also support -N which shows captured output for tests.

Finally, allow selection of a particular test to run.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
10c0f22d51 patman: Tidy up the start-up code
Much of this was written before the other Python tools and they have
evolved. Make a few updates:

- Rather than calling sys.exit(), return the exit code from the control
  module and use it in __main__
- Set up tout as it is used in some places
- We now have quite a few tests, so update the comment about that

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
24396e5314 patman: Rename test_basic() in test_checkpatch
This currently has the same name as a test in func_test.py so it isn't
possible to select one or the other with 'patman test test_basic': both
are executed.

Rename this one to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
77c2d22676 patman: Correct a stray quote
A quote character was added a few years ago, but was not intended. Drop
it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 07:09:55 -06:00
Simon Glass
de40e515d1 tools: Plumb in capture control
Add control of capturing output into u_boot_pylib and the tools which
use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 06:52:36 -06:00
Simon Glass
10b761d023 u_boot_pylib: Allow disabling colour output
When running tests there is no situation in which we want ANSI output as
it makes it much harder to see what is going on in logs, tests, etc.

Provide a way to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 06:52:36 -06:00
Simon Glass
5e0c772739 u_boot_pylib: Avoid concurrent execution of only one test
There is no point in spinning up multiple processes if there is only one
test to execute. Add a check for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 06:52:36 -06:00
Simon Glass
00e82a65f4 u_boot_pylib: Add a pager
It is useful to have a pager when outputting a lot of text. Add support
for this in the terminal library, making use of a context manager.

Also add a function to indicate whether the output device is a terminal
or not, while we are here, to avoid duplicating this code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 06:52:36 -06:00
Simon Glass
0e0aa0e9ad u_boot_pylib: Improve ANSI-colour output with backgrounds
The current implementation does not handle background colours very well:

- It outputs an incorrect code in some cases, leading to wrong colours
- Some functions lack a control for the background

Tidy this up so that background colours can be used in more places.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-29 06:52:19 -06:00
9 changed files with 12 additions and 15 deletions

View File

@@ -107,10 +107,8 @@ mkdir ${dir}/tests
cd ${dir}
# Make sure the tools are up to date
if ! sudo apt install python3-build twine; then
python3 -m pip install --upgrade build
python3 -m pip install --upgrade twine
fi
python3 -m pip install --upgrade build
python3 -m pip install --upgrade twine
# Build the PyPi package
python3 -m build

View File

@@ -10,11 +10,11 @@ authors = [
]
dependencies = ["pylibfdt", "u_boot_pylib >= 0.0.6", "dtoc >= 0.0.6"]
description = "Binman firmware-packaging tool"
license = "GPL-2.0-or-later"
readme = "README.rst"
requires-python = ">=3.7"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)",
"Operating System :: OS Independent",
]

View File

@@ -14,11 +14,11 @@ dependencies = [
"patch-manager >= 0.0.6"
]
description = "Buildman build tool for U-Boot"
license = "GPL-2.0-or-later"
readme = "README.rst"
requires-python = ">=3.7"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)",
"Operating System :: OS Independent",
]

View File

@@ -10,11 +10,11 @@ authors = [
]
dependencies = ["pylibfdt", "u_boot_pylib >= 0.0.6"]
description = "Devicetree-to-C generator"
license = "GPL-2.0-or-later"
readme = "README.rst"
requires-python = ">=3.7"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)",
"Operating System :: OS Independent",
]

View File

@@ -177,7 +177,7 @@ class TestFunctional(unittest.TestCase):
Commit-changes: 2
- Changes only for this commit
' Cover-changes: 4
Cover-changes: 4
- Some notes for the cover letter
Cover-letter:

View File

@@ -10,11 +10,11 @@ authors = [
]
dependencies = ["u_boot_pylib >= 0.0.6", "aiohttp >= 3.9.1" ]
description = "Patman patch manager"
license = "GPL-2.0-or-later"
readme = "README.rst"
requires-python = ">=3.7"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)",
"Operating System :: OS Independent",
]

View File

@@ -9,7 +9,7 @@ collected from patchwork.
"""
import asyncio
import collections
from collections import defaultdict
import concurrent.futures
from itertools import repeat
@@ -42,7 +42,7 @@ def process_reviews(content, comment_data, base_rtags):
list of patchwork.Review: reviews received on the patch
"""
pstrm = patchstream.PatchStream.process_text(content, True)
rtags = collections.defaultdict(set)
rtags = defaultdict(set)
for response, people in pstrm.commit.rtags.items():
rtags[response].update(people)
@@ -57,7 +57,7 @@ def process_reviews(content, comment_data, base_rtags):
rtags[response].update(people)
# Find the tags that are not in the commit
new_rtags = collections.defaultdict(set)
new_rtags = defaultdict(set)
for tag, people in rtags.items():
for who in people:
is_new = (tag not in base_rtags or
@@ -102,7 +102,6 @@ def compare_with_series(series, patches):
warnings.append("Cannot find patch for commit %d ('%s')" %
(seq + 1, cmt.subject))
# Check the names match
commit_for_patch = {}
all_commits = set(series.commits)

View File

@@ -137,7 +137,7 @@ Signed-off-by: Simon Glass <sjg@chromium.org>
class TestPatch(unittest.TestCase):
"""Test the u_boot_line() function in checkpatch.pl"""
def test_basic(self):
def test_filter(self):
"""Test basic filter operation"""
data='''

View File

@@ -9,11 +9,11 @@ authors = [
{ name="Simon Glass", email="sjg@chromium.org" },
]
description = "U-Boot python library"
license = "GPL-2.0-or-later"
readme = "README.rst"
requires-python = ">=3.7"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)",
"Operating System :: OS Independent",
]