[Bug] Error during eksctl utils update-aws-node when updating cluster from 1.30 to 1.31 (eksctl version 0.202.0)

[vandamb]

What were you trying to accomplish?

After successfully updating EKS cluster from 1.30 to 1.31, trying to update the CNI plugin using eksctl utils update-aws-node --approve .

What happened?

The command failed with below error:
Error: DaemonSet.apps "aws-node" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"aws-vpc-cni", "app.kubernetes.io/name":"aws-node"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

How to reproduce it?

Simply run the command eksctl utils update-aws-node --approve

Logs
2025-01-27 15:40:40 [ℹ] replaced "CustomResourceDefinition.apiextensions.k8s.io/eniconfigs.crd.k8s.amazonaws.com"
2025-01-27 15:40:41 [ℹ] replaced "CustomResourceDefinition.apiextensions.k8s.io/policyendpoints.networking.k8s.aws"
2025-01-27 15:40:41 [ℹ] skipped existing "kube-system:ServiceAccount/aws-node"
2025-01-27 15:40:41 [ℹ] replaced "kube-system:ConfigMap/amazon-vpc-cni"
2025-01-27 15:40:41 [ℹ] replaced "ClusterRole.rbac.authorization.k8s.io/aws-node"
2025-01-27 15:40:41 [ℹ] replaced "ClusterRoleBinding.rbac.authorization.k8s.io/aws-node"
Error: DaemonSet.apps "aws-node" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"aws-vpc-cni", "app.kubernetes.io/name":"aws-node"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

Anything else we need to know?
Nothing special, this has worked in previous updates all the time.
Self managed EKS cluster and plugins

Versions
eksctl info
eksctl version: 0.202.0
kubectl version: v1.31.4
OS: linux

[inih]

I'm also experiencing the same bug

eksctl version: 0.202.0
kubectl version: v1.31.2
OS: linux

[vla6t0r]

I've got the same issue when updating EKS from v1.29 to 1.30

eksctl version: 0.202.0
kubectl version: v1.29.2
OS: MacOS

Update: works fine with the previous eksctl version 0.199.0

[nackjicholson]

I too downgraded to 0.201.0 and tried again. Works fine.

[sillyfrog]

Same issue for me running eksctl version 0.203.0
Downgrading to 0.201.0 fixed it

(At the moment I'm using v0.203 to start the upgrade to v1.32, and then v0.201 to do the service updates)

[devops-cafex]

Hitting this same issue with eksctl 0.202.0 needed to downgrade to 0.201.0 to run the service updates

[dims]

@bryantbiggs could it be this change?

c143a9c#diff-00df199028ae76ae35f1678fdb2184bf611cfb868abc937e86828e2aa2ee0150R407-R409

[bryantbiggs]

Could be - @vandamb can you try 0.204.0 which included an automatic update for this in #8160

[bryantbiggs]

Looks like the eksctl manifest labels do not match the upstream. So when using the upstream chart to update here, the labels need to be manually modified

Or better yet, we remove the self-managed addon usage here and simply use the EKS addon API for this

[bryantbiggs]

has anyone been able to try 0.204.0 to confirm that this has been resolved?

[sillyfrog]

Good timing, I've just been testing for the last hour on a couple of clusters and it's looking good!

I've upgraded from v1.30 and v1.31 to v1.32, all using 0.204.0. I upgraded nodegroups and addons (and k8s version), all with no issues.

Thanks!

[bryantbiggs]

Thank you @sillyfrog ! I'll close this one as resolved. If anyone is still encountering errors, please open a new issue with the relevant details