Fix the logic for determining if an MR's last commit is newer than the
current database position. The check should verify that current is an
ancestor of the MR's commit (meaning the MR's commit is newer), not the
other way around.
Add a test to cover this.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Change get_pickman_mrs() and get_mr_comments() to return lists of named
tuples instead of dicts. This provides cleaner attribute access (e.g.
merge_req.iid instead of merge_req['iid']) and better code documentation.
Add PickmanMr and MrComment named tuples to gitlab_api.py.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Extract the core 'apply' logic into execute_apply() which handles
database operations, agent invocation, and MR creation. This separates
the non-git logic from do_apply() which now handles setup, history
writing, and branch restoration.
Also move branch restoration to the end of do_apply() using a ret
variable to ensure we always restore the branch even if MR creation
fails.
Add tests for execute_apply() covering success, failure, push, and
push failure cases.
Rename conversation_log to conv_log for brevity.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Extract the setup logic from do_apply() into prepare_apply() which:
- Gets the next commits from the source branch
- Validates the source exists and has commits
- Saves the original branch
- Generates or uses provided branch name
- Deletes existing branch if needed
- Prints info about what will be applied
Returns (ApplyInfo, return_code) tuple where ApplyInfo contains commits,
branch_name, original_branch, and merge_found. This makes the setup
logic testable independently from the agent and git operations.
Add tests for prepare_apply():
- test_prepare_apply_error: Test error handling for unknown source
- test_prepare_apply_no_commits: Test when no commits to apply
- test_prepare_apply_with_commits: Test successful preparation
- test_prepare_apply_custom_branch: Test custom branch name
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Extract get_history() from write_history() to separate the history
content generation and file I/O from the git operations. The function
now:
- Takes a filename parameter
- Reads existing content from the file
- Writes updated content to the file
- Returns both the content and the git commit message
This makes the history generation logic fully testable without needing
to mock git operations.
Add tests for get_history():
- test_get_history_empty: Test with no existing file
- test_get_history_with_existing: Test appending to existing content
- test_get_history_replaces_existing_branch: Test replacing entry
- test_get_history_multiple_commits: Test commit message with multiple
commits
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add tests to improve test coverage for control.py:
- test_poll_continues_on_step_error: Test poll warning on step error
- test_format_history_summary: Test history summary formatting
- test_get_next_commits_with_empty_lines: Test empty line handling
- test_commit_source_resolve_error: Test commit resolution failure
- test_unknown_command: Test unknown command handling
- test_review_with_mrs_no_comments: Test review with open MRs
This improves control.py coverage from 63% to 67%.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add support for running pickman tests with code coverage checking,
similar to binman. Use test_util.run_test_suites() for running tests
and test_util.run_test_coverage() for coverage checking.
Options added to the 'test' command:
-P: Number of processes for parallel test execution
-T: Run with coverage checking
-v: Verbosity level (0-4)
The coverage check allows failures for modules that require external
services (agent.py, gitlab_api.py, control.py, database.py) since
these cannot easily achieve 100% coverage in unit tests.
Also update test_util.py to recognize 'pickman' as using the 'test'
subcommand like binman and patman.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add tests for the remaining uncovered database.py code paths:
- Duplicate database creation error
- Already open error when opening twice
- Already closed error when closing twice
- rollback() method
This brings database.py to 100% test coverage.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
- Remove unused commit_hashes variable in agent.py
- Add pylint disable for too-many-public-methods in database.py
- Add pylint disable for too-many-arguments in database.py functions
- Add pylint disable for too-many-branches in control.py do_apply()
- Add pylint disable for too-many-lines in ftest.py
- Prefix unused mock arguments with underscore in ftest.py
This brings the pylint score to 10.00/10.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move datetime, re, and time imports from inside functions to the top
of the file, following Python style guidelines.
Also fix a few pylint warnings while here.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add terminal.capture() context manager to tests that may produce
terminal output, ensuring tests run silently unless explicitly
checking output content.
Fix a few pyline warnings while here.
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Have the step command also process review comments when there are open
MRs. This integrates the review functionality into step, so a single
poll loop can handle the full workflow: process merged MRs, handle
review comments, and create new MRs when ready.
Refactor process_mr_reviews() as a helper function used by both
do_review() and do_step().
Series-to: concept
Series-cc: heinrich
Cover-letter:
pickman: Add a manager for cherry-picks
This series adds pickman, a tool to help manage cherry-picking commits
between branches. It's designed to automate the tedious process of
cherry-picking large numbers of commits from one branch to another,
such as when maintaining a downstream branch.
Key features:
- Track source branches and cherry-pick progress in a SQLite database
- Find commits to cherry-pick up to the next merge commit
- Use Claude AI agent to automate cherry-picks with conflict resolution
- Create GitLab merge requests for review
- Handle MR review comments automatically
- Poll for continuous automated cherry-picking
The tool integrates with GitLab for MR management and uses the Claude
Agent SDK for automated cherry-picking and conflict resolution.
This tool is *very* experimental (so don't use it yourself!), but should
become usable sometime in the new year.
END
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add automatic database updates when pickman MRs are merged. The step
command now:
1. Checks for merged pickman MRs via GitLab API
2. Parses the MR description to find the source branch and last commit
3. Updates the database's last_commit for the source branch
4. Then proceeds to check for open MRs and create new ones as before
This removes the need to manually run commit-source after each MR is
merged, enabling fully automated cherry-pick workflows with poll.
Add get_merged_pickman_mrs() and refactor get_open_pickman_mrs() to use
a common get_pickman_mrs() function with a state parameter.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Add a workflow section at the top of the README explaining the typical
usage pattern for pickman: setup with add-source, cherry-pick with
apply -p, review/merge, update with commit-source, and repeat.
Also mention the step and review commands for automated workflows.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add do_step() command that checks for open pickman MRs first. If none
exist, it runs apply with --push to create a new one. This enables
automated cherry-picking workflows where only one MR is active at a time.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add do_review() command that lists open pickman MRs and checks each for
human comments. Uses a Claude agent to address actionable comments.
Also add supporting infrastructure:
- gitlab_api: get_open_pickman_mrs(), get_mr_comments(), reply_to_mr()
- agent: run_review_agent(), handle_mr_comments()
- History file support for recording cherry-pick conversations
- Return conversation_log from cherry_pick agent for MR descriptions
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a command to update the database with the last cherry-picked commit.
This allows reviewing the cherry-picked branch before committing to the
database.
Usage:
pickman commit-source us/next <commit-hash>
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add -p/--push option to the apply command to push the cherry-pick branch
to GitLab and create a merge request. Uses the python-gitlab library.
Options:
-p, --push Push branch and create GitLab MR
-r, --remote Git remote for push (default: ci)
-t, --target Target branch for MR (default: master)
Requires GITLAB_TOKEN environment variable to be set.
Also record cherry-pick history in .pickman-history file on successful
apply. Each entry includes the date, source branch, commits, and the
agent's conversation log. This file is committed automatically and
included in the MR description when using -p.
Name the module gitlab_api.py to avoid shadowing the python-gitlab
library.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add gitlab_api module with functions for interacting with GitLab:
- check_available(): Check if python-gitlab is installed
- get_token(): Get GitLab API token from environment
- get_remote_url(): Get URL for a git remote
- parse_url(): Parse GitLab URLs (SSH and HTTPS formats)
- push_branch(): Push a branch to a remote
- create_mr(): Create a merge request via GitLab API
- push_and_create_mr(): Combined push and MR creation
Requires python-gitlab library and GITLAB_TOKEN environment variable.
Name the module gitlab_api.py to avoid shadowing the python-gitlab
library.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a command that automates cherry-picking commits using the agent
module. The apply command:
- Gets the next set of commits (same as next-set)
- Creates a new branch for the cherry-picks (-b/--branch option)
- Runs the agent to execute the cherry-picks
- Returns to the original branch after completion
The database is not updated automatically; use 'commit-source' to update
after reviewing the cherry-picked branch.
Also refactor control.py to extract get_next_commits() for reuse and
use a dispatch table for commands.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add an agent which uses the Claude Agent SDK to automate cherry-picking
commits.
The agent module provides an async interface to the Claude Agent SDK
with a synchronous wrapper for easy use from the CLI.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add new database tables to track individual commits and merge requests:
- pcommit: tracks commits being cherry-picked with status, author,
subject, and optional link to merge request
- mergereq: tracks GitLab merge requests with branch name, MR ID,
status, and URL
Also add helper functions for both tables and update control.py to
write commit status to the database during apply operations.
Update README.rst with documentation for all database tables.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a command that finds the next set of commits to cherry-pick from a
source branch. It lists commits from the last cherry-picked commit up
to and including the next merge commit, which typically represents a
logical grouping (e.g., a pull request).
If no merge commit is found, it lists all remaining commits with a note
indicating this.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a command to list all tracked source branches and their last
cherry-picked commits from the database.
Usage: ./tools/pickman/pickman list-sources
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a command to register a source branch in the database. This finds
the merge-base commit between master and the source branch and stores
it as the starting point for cherry-picking.
Usage: ./tools/pickman/pickman add-source <branch>
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add an sqlite3 database module to track the state of cherry-picking
commits between branches. The database uses .pickman.db and includes:
- source table: tracks source branches and their last cherry-picked
commit into master
- Schema versioning for future migrations
The database code is mostly lifted from patman
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add subcommand support:
- compare: Compare branches (existing functionality)
- test: Run the functional tests
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a tool to check the number of commits in a source branch
(us/next) that are not in the master branch (ci/master), and find
the last common commit between them.
Co-developed-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Update the documentation to cover the new options and features:
- Add --kloc and --html options to dirs command options list
- Add new "HTML Reports" section explaining --html usage
- Note that files are sorted alphabetically and hidden when zero active
- Update all example outputs to reflect current behavior
Cover-letter:
codman: Assorted improvements and HTML output
This series adds various improvements to codman including:
- Hide files with zero active lines by default
- Sort files alphabetically and simplify output columns
- Better formatting with kLOC display and whole number percentages
- HTML report generation with collapsible hierarchical drill-down
END
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Add HTML output support with collapsible hierarchical directory
structure, color-coded metrics, and build info in the header.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Raw line counts can be hard to read for large codebases. Add a --kloc
(-k) option to display line counts in kilolines format for a more
compact view.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
When showing files within directories (dirs -fs), there is no visual
separation between one directory's files and the next directory. Add
a blank line after each file list to improve readability.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
File percentages show one decimal place while directory percentages
show whole numbers. Display file %Code as whole numbers to match the
directory format for cleaner, more consistent output.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
File metrics do not align with directory columns, making it hard to
compare them visually. Align file output with directory columns by:
- Skipping the Files, Used, and %Used columns (leave empty)
- Showing percentage active in the %Code column
- Showing total lines (in kLOC) in the kLOC column
- Showing active lines (in kLOC) in the Used column
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Files appear in arbitrary order, making it hard to find specific
files. Sort them alphabetically by filename for easier navigation.
Also remove the inactive lines column from file output to reduce
clutter, keeping only: filename, total lines, active lines, and
percentage active.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Files with 0 active lines clutter the output when viewing directories
with 'dirs -fs'. Hide them by default, matching the directory display
behavior. Use the -e flag (dirs -fse) to show them.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Provide a description of the purpose of codman and some examples of how
to use it.
Series-to: concept
Cover-letter:
codman: Add a new source-code analysis tool
Add a new tool called 'codman' (code manager) for analysing source code
usage in U-Boot builds. This tool determines which files and lines of
code are actually compiled based on the build configuration.
The tool provides three analysis methods:
- unifdef: Static preprocessor analysis (default)
- DWARF: Debug information from compiled code (-w)
- (experimental) LSP: Language server analysis using clangd (-l)
Codman supports:
- File-level analysis: which files are compiled vs unused
- Line-level analysis: which lines are active vs removed by preprocessor
- Kconfig-impact analysis with -a/--adjust option
- Various output formats: stats, directories, detail, summary
Since there is quite a lot of processing involved, Codman uses parallel
processing where possible.
This tool is admittedly not quite up to my normal code quality, but it
has been an interesting experiment in using Claude to create something
from scratch.
The unifdef part of the tool benefits from some patches I created for
that tool:
- O(1) algorithm for symbol lookup, instead of O(n) - faster!
- support for IS_ENABLED(), CONFIG_IS_ENABLED()
Please get in touch if you would like the patches.
This series also includes a minor improvement to buildman and a tidy-up
of the tout library to reduce code duplication.
END
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Series-links: 1:65
It is possible to use an LSP to determine which code is used, at least
to some degree.
Make a start on this, in the hope that future work may prove out the
concept.
So far I have not found this to be particularly useful, since it does
not seem to handle IS_ENABLED() and similar macros when working out
inactive regions.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a way to do static preprocessor analysis using debug information
from compiled code. This reads the DWARF tables to determin which lines
produced code.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a way to do static preprocessor analysis using unifdef, as a way of
figuring out what code is actually used in the build.
I have modified the unifdef tool as follows:
- O(1) algorithm for symbol lookup, instead of O(n)
- support for IS_ENABLED(), CONFIG_IS_ENABLED()
The first patch was sent upstream. The others are U-Boot-specific so I
have not submitted those.
Please get in touch if you would like the patches.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add a new tool called 'codman' (code manager) for analysing source code
usage in U-Boot builds. This tool determines which files and lines of
code are actually compiled based on the build configuration.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Allow users to specify multiple config adjustments in a single -a
argument using commas. This is more convenient than repeating -a
multiple times.
Examples:
buildman -a FOO,~BAR
buildman -a FOO,~BAR -a BAZ=123
Add tests to verify comma-separated values work correctly.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Refactor tout.py to use terminal.tprint() instead of direct print()
calls. This provides better control over output formatting and
supports the new stderr parameter. It also reduces code duplication.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Add optional stderr parameter to tprint() to allow printing to stderr
instead of stdout.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Change the indent field in fit_print_ctx from a string pointer to an int
number of spaces to indent.
Set the initial indent value to 3 to match IMAGE_INDENT_STRING
Drop indentation from the debug() calls since these are not visible to
users.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Move the indent string into struct fit_print_ctx so it is available to
the printing functions. This avoids having to pass it as a separate
parameter.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Create a struct fit_print_ctx to hold the FIT pointer and pass it to all
printing functions instead of passing the FIT pointer directly. This
provides a foundation for adding additional context in the future.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
There is enough code here that it makes sense to put it in its own file.
Create a new fit_print.c file, before undertaking future refactoring.
Printing is only included in the main build if CONFIG_FIT_PRINT is
enabled, although it is always included in the tools build.
Add static inlines for existing callers.
Make a few small code-style adjustments, including fixing checkpatch
warnings about over-use of brackets.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
Remove the unused third parameter (len) from fit_get_name(). All uses of
this function pass NULL for this parameter.
Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
