feature request: detect unused dependency lists

[jameslamb]

Description

rapids-dependency-file-generator should detect unused dependency lists, and it should be possible to opt-in to the presence of such lists resulting in a non-0 exit code.

Consider a dependencies.yaml like the following:

files:
  all:
    output: conda
    matrix:
      arch: [x86_64, aarch64]
    includes:
      - run
channels:
  - conda-forge
dependencies:
  run:
    common:
      - output_types: [conda]
        packages:
          - pandas
  depends_on_cugraph_pyg:
    common:
      - output_types: [conda]
        packages:
          - cugraph-pyg

That depends_on_cugraph_pyg is unreachable... it isn't referenced anywhere in files:, and rapids-dependency-file-generator's public API doesn't support any access to individual items in dependencies:.

Such things are sometimes left behind during refactorings... it would be awesome if the rapids-dependency-file-generator pre-commit hook could catch them.

Benefits of this work

• simplifies dependencies.yaml, via removal of unnecessary configuration
• avoids unnecessary work updating useless lists (as I started to do add -cu13 equivalents for every -cu12 thing I found for the CUDA 13 rollout in Build and test with CUDA 13.0.0 cugraph#5236)

Acceptance Criteria

• rapids-dependency-file-generator (including the pre-commit hook) can be configured to fail (non-0 exit code) on the presence of unreachable lists in dependencies:

Approach

To be safe, this behavior should probably be opt-in in the CLI. For example, it should be possible to include lists that look "unused" with TODO comments to integrate them in later commits.

Flag name ideas:

• --strict (which could maybe be expanded to more things like idea: linting dependencies.yaml files via '--strict' #103 or Raise on duplicate keys #104 in the future)
• --fail-on-unused-dependencies

I kind of like the idea of a specific-to-this flag name, and maybe later (as a separate thing), introducing a --strict which implies enabling multiple flags. Like mypy has done: https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-strict

Notes

Just opening this to document it, I'm not planning to work on it any time soon. Anyone is welcome to take this up if it interests them.

Inspired by rapidsai/cugraph#5237

[GPUtester]

🎉 This issue has been resolved in version 1.20.0 🎉

The release is available on:

• GitHub release
• PyPI release
• Conda release - rapidsai
• Conda release - rapidsai-nightly

Your semantic-release bot 📦🚀