-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Description
What were you trying to accomplish?
Upgrading a cluster to 1.30 with managed node groups that set maxPodsPerNode. Presumably the same issue would occur creating a new cluster on 1.30 with the same config.
What happened?
eksctl produces the following error:
Error: eksctl does not support configuring maxPodsPerNode EKS-managed nodes based on AmazonLinux2023
which comes from the validation here.
How to reproduce it?
Create a ClusterConfig for Kubernetes 1.30 or higher with a managed node group that sets maxPodsPerNode to some value above zero.
Versions
$ eksctl info
eksctl version: 0.194.0
kubectl version: v1.31.1
OS: linux
Further comments
Having dug through the code a little bit, it seems that the reason this was added was because AL2023 EKS-managed nodes have their user data automatically populated to include a NodeConfig that sets maxPods: de74c5b#r1546826066
I'm slightly confused by this reasoning, however:
- nodeadm supports merging multiple
NodeConfigs from a set of userdata (ref). - From what I can tell, EKS inserts its
NodeConfiginto the user data of an instance above the user data specified by the template. - Looking at the codepath for merging NodeConfigs, NodeConfigs get merged top-to-bottom from the MIME multi-part user data.
Therefore, I think that this means any NodeConfig specified in the launch template's user data should get merged with and override equivalent fields of the (auto-generated) NodeConfig that gets inserted into the instance user data by EKS, and hence that we can override maxPods.
If the reasoning above is sound, I think this would be a reasonably straightforward fix: one could add some logic into createNodeConfg() in pkg/nodebootstrap/al2023.go to return a basic NodeConfig that sets maxPods (omitting the cluster details as these would get set by EKS).
On the other hand, I may be missing a deeper understanding of why this restriction was added in the first place. If the maintainers believe this reasoning is sound, I'm happy to open a PR!