fix: Allow west commands to be imported from a project subdirectory if manifest is located in subdirectory by nmunnich · Pull Request #920 · zephyrproject-rtos/west

nmunnich · 2026-02-16T22:18:32Z

Fixes #725

Note that I relied on the test to tell me if the fix is working, I did not test it with a proper repository setup. Let me know if this is desired/this project is one where manual testing of such kind is necessary.

I'm unfamiliar with west release cycles - if this is accepted, could someone let me know when I could expect to see a new version of west released, so that we can start recommending setups relying on this to our users?

nmunnich · 2026-02-16T23:39:10Z

Should pass the linter this time (sorry!)

marc-hb

Thank you so much for taking this! See comments below.

Sorry if some comments appear twice, I had to make them again because you force pushed in the mean time. Please keep force-pushing, it's just Github support for force-pushes is pretty bad.

tests/test_manifest.py

src/west/manifest.py

nmunnich

Thanks for the quick review! (I hope the linter is happy with me this time, uv run poe all doesn't seem to check whatever that last test is)

src/west/manifest.py

tests/test_manifest.py

src/west/manifest.py

tests/test_manifest.py

src/west/manifest.py

tests/test_manifest.py

src/west/manifest.py

nmunnich

Ah, the formatter does run, but for whatever reason it thinks it needs to format 22 files (with no difference made to them) on my machine. I've kept that formatting on manifest.py, maybe now it'll be happy? Just another quirk of Windows I suppose.

tests/test_manifest.py

src/west/manifest.py

codecov · 2026-02-17T14:07:32Z

Codecov Report

❌ Patch coverage is 81.25000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.92%. Comparing base (548b9ee) to head (51c2f44).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/west/manifest.py	81.25%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #920      +/-   ##
==========================================
- Coverage   85.95%   85.92%   -0.03%     
==========================================
  Files          11       11              
  Lines        3453     3461       +8     
==========================================
+ Hits         2968     2974       +6     
- Misses        485      487       +2

Files with missing lines	Coverage Δ
src/west/manifest.py	`95.36% <81.25%> (-0.16%)`	⬇️

nmunnich · 2026-02-17T16:07:49Z

Hmmm. I'm not sold on adding tests to test that the exceptions are indeed raised. Don't think the AssertionError can ever be raised anyway, it's just good practice to have it there.

marc-hb

Hmmm. I'm not sold on adding tests to test that the exceptions are indeed raised. Don't think the AssertionError can ever be raised anyway, it's just good practice to have it there.

I agree. Error-handling is notoriously lacking coverage across the entire industry (one of the reasons software is so insecure), but I don't think we need test coverage for raise InternalBug("this should never happen").

@pdgendt is big on test coverage and rightly so, let's wait until next week when he is back. Every PR needs 2 approvals anyway.

tests/test_manifest.py

src/west/manifest.py

Use backwards slashes on Windows to catch any future hardcoding mistake. Minor oversight found while discussing zephyrproject-rtos#920

Copilot

Pull request overview

This pull request fixes issue #725 where imported west-commands from project subdirectories were not resolving correctly. When a manifest file is located in a subdirectory of a project (e.g., app/west.yml), and that manifest defines custom west-commands, the paths to those commands need to be resolved relative to the manifest's location, not the project root.

Changes:

Modified _import_path_from_project to pass the manifest path instead of None to _import_data_from_project
Updated _import_data_from_project to accept both _import_map and str types using match/case syntax, extracting the manifest directory and prepending it to west_commands paths
Added a test case to verify that west-commands are correctly resolved when imported from project subdirectories

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
src/west/manifest.py	Updated import logic to correctly resolve west-commands paths relative to manifest subdirectory location
tests/test_manifest.py	Added test case to verify west-commands resolution from project subdirectories

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-19T21:24:39Z

src/west/manifest.py

+                imap_path_prefix = '.'
+                mfst_path = imap_or_mfpath
+            case _:
+                raise AssertionError(f'imap_or_path has unexpected type {type(imap_or_mfpath)}')


The error message refers to 'imap_or_path' but the parameter name is 'imap_or_mfpath'. The error message should be updated to match the parameter name for consistency.

Suggested change

raise AssertionError(f'imap_or_path has unexpected type {type(imap_or_mfpath)}')

raise AssertionError(f'imap_or_mfpath has unexpected type {type(imap_or_mfpath)}')

Copilot · 2026-02-19T21:24:39Z

tests/test_manifest.py

 def test_import_map_error_handling():
    # Make sure we handle expected errors when loading import:
    # values that are maps.


While the new test covers string path imports (e.g., import: "subdir/west.yml"), it would be beneficial to also add a test case for import maps with subdirectories (e.g., import: {file: "subdir/west.yml"}). Although both code paths converge at the same logic in _import_data_from_project, having explicit test coverage for both import styles would ensure the fix works correctly in all scenarios and prevent potential regressions.

Suggested change

def test_import_map_error_handling():

# Make sure we handle expected errors when loading import:

# values that are maps.

def test_import_project_submanifest_commands_from_project_subdirectory_import_map(

manifest_repo,

):

# Similar to test_import_project_submanifest_commands_from_project_subdirectory,

# but this tests using an import map with a 'file' key instead of a string path.

# When a manifest is imported from a project subdirectory (e.g., mf_subdir/west.yml),

# and that manifest defines west-commands, the paths should be resolved relative to

# the manifest subdirectory. This tests _import_path_from_project with an import map.

with open(manifest_repo / 'west.yml', 'w') as f:

f.write('''\

manifest:

projects:

- name: p1

url: url-placeholder

import:

file: mf_subdir/west.yml

''')

p1 = manifest_repo / '..' / 'p1'

create_repo(p1)

create_branch(p1, 'manifest-rev', checkout=True)

add_commit(

p1,

'add mf_subdir/west.yml with west-commands (import map)',

files={

'mf_subdir/west.yml': '''\

manifest:

projects:

- name: p2

url: url-placeholder2

self:

west-commands: p2subdir/west-commands.yml

''',

},

)

checkout_branch(p1, 'master')

p1_proj = MF().get_projects(['p1'])[0]

# The west_commands path should be 'mf_subdir/p2subdir/west-commands.yml',

# not 'west-commands.yml', to be resolved correctly relative to the project root.

expected = ['mf_subdir/p2subdir/west-commands.yml']

for a, e in zip(p1_proj.west_commands, expected, strict=True):

assert PurePath(a) == PurePath(e)

def test_import_map_error_handling():

# Make sure we handle expected errors when loading import:

# values that are maps.

That's an interesting suggestion. But that's also a MASSIVE duplication of the other test code for very little additional coverage :-( @nmunnich maybe you could easily combine both tests (yours and the copilot ~duplicate) into a single test like this:

manifest: projects: - name: pA url: url-placeholderA import: mf_subdirA/west.yml - name: pB url: url-placeholderB import: file: mf_subdirB/west.yml

Then you can re-use the identical add_commit() twice in both A and B.

Copilot · 2026-02-19T21:24:40Z

src/west/manifest.py

+        # (manifest_path is a relative path within the project),
+        # we need to adjust the west_commands paths to be relative
+        # to the project root, not to the manifest subdirectory.
+        mfst_dir = Path(mfst_path).parent


Path should be PurePosixPath to ensure cross-platform compatibility. Git stores paths internally using POSIX-style forward slashes, even on Windows. Using Path on Windows will create paths with backslashes, which may not work correctly. The codebase already uses PurePosixPath in similar contexts (see line 312).

Suggested change

mfst_dir = Path(mfst_path).parent

mfst_dir = PurePosixPath(mfst_path).parent

@marc-hb maybe related to #921 ?

Yes, related to #921 (which I just demoted to draft).

On one hand we should stick to Pathlib abstractions, avoid hardcoding forward slashes and not use Unix paths objects internally when running on Windows. This is not just about backslashes, there are other differences.

On the other hand we don't want west manifest --resolve to convert forward slashes to backslashes on Windows.

I need again more time to think about it and do more testing, sorry :-(

This is a much bigger can of worms than I expected. I wish I could "unsee" it but I can't :-(

For instance, on Windows:

west list -f '{path}' # forward slashes west list -f '{abspath}' # backslashes west list -f '{posixpath}' # forward slash west list topdir # absolute and forward since #375

https://docs.zephyrproject.org/latest/develop/west/west-apis.html#west.manifest.Project

Also relevant:

Overhaul path handling #411 and more

Personally, I would argue that resolving that inconsistency is out of scope here, and I would prefer not to block waiting for a fix/resolution. Would you be alright with taking the PurePosixPath suggestion and going forward with that here?

I'm not sure it's a pre-existing inconsistency. This could be all by design - which this PR may or may not be consistent with.

So, there is some pre-existing inconsistency. But some of it is right in the middle of this PR. Example:

Input:

- name: hal_espressif path: modules///hal\\\espressif west-commands: west\\\west-commands.yml

west manifest --resolve output:

- name: hal_espressif path: modules/hal/espressif # normalized to POSIX west-commands: west\\\west-commands.yml # unchanged

We need to decide at the very least whether this pre-existing behavior is correct or not before merging this PR. I mean we can't modify code without even knowing whether the modified code is buggy or not.

Note this is only the output of west manifest --resolve. How does the actual import behave? On Windows and Linux?

I've gone through the pain of doing a bunch of testing of actual imports in PowerShell. This may be simpler than we think:

No matter what path is set to, it always resolves in west manifest --resolve with /. So path: modules/\//\/\/\\\\/\//\./\zmk\/\/\/\.\\\/\test turned into modules/zmk/test.

No matter what west-commands is set to, it remained the same in the west manifest. In addition, regardless of how many extra \ and / combinations I put, my dummy west command was being picked up and executed correctly.

I've now modified the code somewhat, so that the west-commands will always resolve with posix paths, for consistency with path. I noticed a comment on L2468 that seems relevant to the conversation, and took the approach used there.

I did notice another bug while doing my testing:

manifest: projects: - name: p1 url: placeholder revision: west-test import: app/west.yml west-commands: app/scripts/west-commands.yml

west-commands: - file: test/west-commands/west.py commands: - name: west class: West help: Print hello world to the console

When resolving this, while the west-commands.yml file is located correctly and the commands are visible in west help, trying to actually execute the command doesn't work because west is trying to resolve it as <module-path>/test/west-commands/west.py rather than <module-path>/app/test/west-commands/west.py. My current thinking is that we need to add manifestdir as a property to Project, so that we can refer to it on L691 of commands.py. That feels like a rather extensive change though, would you have any alternative idea?

I've been working on adding some 2 "backslash" tests related to this. I have this one ready but I want to finish a similar one for west-commands:
https://github.com/marc-hb/west/commits/backslash-tests/

I should be able to get the last one ready on Friday. I would like these "pure" tests to be merged before any change (feature or bug fix) in west itself.

Just a heads-up, sorry for not answering your comment yet - will do that on Friday too.

I would like these "pure" tests to be merged before any change (feature or bug fix) in west itself.

Add test coverage for special characters in manifest and west commands #928

backslashes :-(

…ories Allow west commands to be imported from a project subdirectory if the manifest is located in a subdirectory, e.g. `import: mf_subdir/west.yml`. Fixes issue zephyrproject-rtos#725

Detect any unexpected changes in the way we've been handling backslashes and multiple slashes in paths. Changes in how we handle such edge cases may or may not be desired (and this test may be updated accordingly), but we never want these changes to come as a surprise and we want to keep control over them. This came up as part of the review for zephyrproject-rtos#920 which fixes zephyrproject-rtos#725

Add some test coverage for forward and backslashes + whitespace in filenames for west extensions Spurred by a discussion in the review of zephyrproject-rtos#920 which fixes zephyrproject-rtos#725

marc-hb · 2026-03-03T17:06:52Z

src/west/manifest.py

            submodules=self._load_submodules(pd.get('submodules'), f'project {name}'),
            clone_depth=pd.get('clone-depth'),
-            west_commands=pd.get('west-commands'),
+            west_commands=Path(pd.get('west-commands')).as_posix() if pd.get('west-commands') else None,


This normalizes west-commands: values for projects but not for self which seems to come from somewhere else :-(

If we go for west-commands normalization, then maybe just change the type to a Path across the board which can help catch inconsistencies.

Add some test coverage for forward and backslashes + whitespace in filenames for west extensions Spurred by a discussion in the review of zephyrproject-rtos#920 which fixes zephyrproject-rtos#725

marc-hb · 2026-03-03T18:28:50Z

As discussed previously, this PR breaks my new tests submitted in #928. It's probably OK and these tests in #928 can likely be adjusted but it needs to be discussed because it's still a technically backwards-incompatible change. The easiest way to discuss and to record that behavior change is to merge #928 to fill the test coverage gap :-)

marc-hb · 2026-03-03T18:34:27Z

One more thought: this needs to be split into at least 2 commits, maybe 2 PRs:

the west-commands Path normalization
the prefix bug fix

Detect any unexpected changes in the way we've been handling backslashes and multiple slashes in paths. Changes in how we handle such edge cases may or may not be desired (and this test may be updated accordingly), but we never want these changes to come as a surprise and we want to keep control over them. This came up as part of the review for zephyrproject-rtos#920 which fixes zephyrproject-rtos#725 Signed-off-by: Marc Herbert <[email protected]>