Fix false-positive RequiredFilesExist lint in shed_lint (#1646)#1647
Merged
jmchilton merged 1 commit intoMay 26, 2026
Merged
Conversation
galaxy-tool-util 26.0 added a `RequiredFilesExist` linter that walks the tool directory via `safe_walk` to verify every `<required_files>` include resolves to a real file. `safe_walk` silently drops symlinks whose realpath points outside the walked directory, so every file that `shed_lint` had symlinked into its temp "realized" repository was filtered out and reported as missing (galaxyproject#1646). The realized directory is meant to mirror the eventual shed tarball, and `tarfile.add` dereferences symlinks anyway, so copying is the semantically correct operation here.
Member
Author
|
Alternative to galaxyproject/galaxy#22759 |
jmchilton
approved these changes
May 26, 2026
This was referenced May 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Galaxy 26.0 added a
RequiredFilesExistlinter that walks the tool directory viagalaxy.util.path.safe_walkto verify every<required_files>include resolves to a real file.safe_walksilently drops symlinks whose realpath points outside the walked prefix (a deliberate symlink-escape guard).shed_linthad been symlinking files into its temp "realized" repository, so every required-file check now returns empty and falsely reports the file as missing.Fix: in
RealizedFile.realize_to, copy files withshutil.copy2instead ofos.symlink. The realized directory is meant to mirror the eventual shed tarball, andtarfile.adddereferences symlinks anyway, so this brings the on-disk representation in line with both the tarball and the install dir.Fixes #1646.
Why this rather than relaxing
safe_walk?The obvious alternative was to make planemo or the linter walk with symlinks followed (or pass an allowlist that whitelists the original source tree). I deliberately avoided that:
safe_walk's symlink-escape guard exists precisely so a tool directory cannot reach files outside itself, and weakening that here would diverge planemo's view of a tool from Galaxy's.RequiredFilesExist(or any other path-sensitive linter) would hide real bugs — the whole point of the new linter is to catch missing referenced files before they ship.tool_sourcefrom the original source path (instead of the realized one) would bypass the implicit-ignore filtering that the realized repo is specifically there to enforce, and would duplicate tool-source resolution.Copying instead of symlinking keeps Galaxy's safety model intact, keeps the linter active, and accurately reflects what will land in the tarball.
Test plan
tests/test_shed_lint.py::ShedLintTestCase::test_tool_linting_required_files+ fixturetests/data/repos/single_tool_required_files/(tool XML with<required_files><include path=\"helper.py\"/></required_files>next to the helper).ERROR (RequiredFilesExist): Required file [helper.py] does not exist).tests/test_shed_lint.py,tests/test_shed_upload.py,tests/test_shed_create.pyruns clean (the two*_from_gitfailures I saw locally are pre-existing GPG-signing env issues, identical on master).