Disable GPU resource processor for nodes using DRA for accelerator attachment

[mtrqq]

What type of PR is this?

/kind bug

What this PR does / why we need it:

GpuCustomResourcesProcessor.FilterOutNodesWithUnreadyResources filters out node objects with GPU labels while allocatable and capacity properties remain not populated. This is a good readiness indicator for nodes using device plugin, but for DRA-enabled nodes it doesn't work as they don't have device information in the capacity or allocatable. Instead there's another processor which needs to be configured for DRA infrastructure introduced as part of #8109. This CL effectively disables filtering for DRA-enabled nodes within GPU custom resource processor while doesn't affect other places using GPULabel instead of GetNodeGpuConfig

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Nodes with GPUs exposed via DRA are no longer treated as unready if they don't have the nvidia.com/gpu custom resource in allocatable

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[mtrqq]

/assign towca

[towca]

There are a couple more places where we assume that GPU == device plugin:

• GetGpuInfoForMetrics produces different metrics if ResourceName is missing from Node capacity, and returns the ResourceName to be used as a metric label. We need to skip the capacity checking in the DRA case, and return something DRA-specific to be used as the metric label. Something like dra_<dra_driver_name> would make sense. The name of the DRA Nvidia GPU driver is gpu.nvidia.com, but it feels weird hardcoding it in GetGpuInfoForMetrics. Maybe instead of adding AttachedUsingDra we add DraDriverName and rename ResourceName to DevicePluginResourceName? Only one of them would ever be non-empty, and that could be checked instead of the bool. WDYT?
• utilization.Calculate() assumes that if *GpuConfig is non-nil it should base the utilization on ResourceName from Node capacity. This should be a very easy fix, we just need to check the new bool/DRA driver name and go to the DRA util logic if it's true/non-empty.

Could you handle these parts as well? Can be in in further commits in this PR, or further PRs.

[towca]

Thanks for taking care of this! Just have 2 small comments before we can merge

[towca]

Thanks for addressing the comments!

/lgtm
/approve

[towca]

/release-note-edit

Nodes with GPUs exposed via DRA are no longer treated as unready if they don't have the nvidia.com/gpu custom resource in allocatable

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mtrqq, towca

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• cluster-autoscaler/OWNERS [towca]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jackfrancis]

@mtrqq @towca are there any safe ways for us to implement this generally across providers, or is this necessarily a provider-specific + dra driver-specific story?

cc @nojnhuh see above how GCE is implementing disambiguation between device-plugin and DRA driver accelerator-enabled nodes

[mtrqq]

I think we may do something more generic than this, but it'll still be intermediate step before DRA Extended Resource KEP lands and may become de facto a new standard completely replacing device plugin. So at this point it feels that it's not worth investing into any cross cloudprovider refactoring

WDYT, @towca?

[towca]

We discussed this during the SIG Autoscaling meeting, the conclusion was that detecting the enablement should be provider-specific, at least for the foreseeable future.

[jackfrancis]

/cherry-pick cluster-autoscaler-release-1.34

[k8s-infra-cherrypick-robot]

@jackfrancis: new pull request created: #9045

Details

In response to this:

/cherry-pick cluster-autoscaler-release-1.34

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.