Implement judicious skipping of CI jobs

[vyasr]

Currently any change to any RAPIDS repo triggers a complete run of the entire build and test suite of CI jobs. This is very expensive, and is often unnecessary. RAPIDS libraries are typically structured such that they have a clear, linear dependency chain between different components. Some examples:

• Changes to Python code should have no effect whatsoever on C++ testing
• For repositories that publish multiple Python packages, there is typically a linear dependency between at least some of these such that changes to downstream packages has no effect on the tests of upstream package (e.g. changes to cugraph would not affect pylibcugraph tests).
• Changes to documentation should only require documentation rebuilds, no test runs.

We can reduce the number of unnecessary jobs that we run in CI by more judiciously skipping jobs that are unnecessary. The simplest approach to do this is by simply checking what files have changed. We have previously implemented a form of this in cudf for both cudf.pandas and now for cudf-polars. To do this, I would propose the following steps:

• Generalizing the above logic for detecting changes into a shared workflow
• Enable filtering jobs using the above shared workflow
• Generalize the pr-builder job to allow some jobs to be skipped under appropriate circumstances.

[KyleFromNVIDIA]

pytest-incremental may help us with skipping unneeded Python tests:

https://pypi.org/project/pytest-incremental/

It hasn't been updated in a while though, it may need some work.

[vyasr]

@KyleFromNVIDIA now that we have a common shared workflow and about a month of data on how pruning the jobs works, do we want to roll this feature out to the rest of RAPIDS and then close this issue?