[Feature] I would like to be able to setup Karpenter for an existing cluster which I created via EKSCTL

[AndrewFarley]

What feature/behavior/change do you want?

I would like to be able to add-in Karpenter to an existing cluster which I created with EKSCTL. It feels like leaving everyone out in the cold to have to setup/manage something externally, when I feel like with much less effort EKSCTL could/should be able to handle this. I'm proposing something like the following...

1. To an existing cluster's configuration file, add the necessary values, eg:

  tags:
    karpenter.sh/discovery: eks-dev
karpenter:
  version: '0.9.0'

2. Add a new command to setup Karpenter on this existing cluster to have a clear and distinct code path to follow which is to setup Karpenter on an existing EKSCTL-setup cluster. Such as...

eksctl create karpenter
or
eksctl enable karpenter

# I would imagine you would also need the opposite, incase people want to stop using it...
eksctl disable karpenter

Why do you want this feature?

1. Because it seems silly to have to manually set something up because we didn't happen to create our clusters recently
2. Because it would be far less effort for the community to have EKSCTL take ownership over this responsibility indefinitely, similar to how it takes ownership and allows management over the nodegroups we deploy via EKSCTL
3. Because it seems silly to have to create a whole stack of automation for IaC (eg: in Terraform) just for this component which EKSCTL already does, but currently only supports it during cluster creation.

Reference: #5380

^ Clearly I'm not the only one who wants this, I imagine as Karpenter's popularity and stability grows you'll have more and more people wanting to adopt it. Why not make it easy(ier)?

[hassaanakram]

Hey @Himangini
I'd like to take a stab at it because I share Andrew's opinion. However, before doing so I would like to know if this is a direction that eksctl maintainers would like to go for? This way the eksctl enable code path may be further expanded to include more common functionality.

Would love to hear your thoughts. Thanks!

[Himangini]

Hey @Himangini I'd like to take a stab at it because I share Andrew's opinion. However, before doing so I would like to know if this is a direction that eksctl maintainers would like to go for? This way the eksctl enable code path may be further expanded to include more common functionality.

Would love to hear your thoughts. Thanks!

@hassaanakram thanks for your interest in this issue. This is something on our radar and we're looking for making improvements in this area. However, we can't offer any concrete plans at this point. We'll post more updates this year, watch this space ✨

[maxxd]

I would prefer to manage the karpenter Helm release myself, but the AWS tags, IAM resources, and groups make more sense from eksctl, and it would certainly be useful to be able to update these on an already-created eksctl cluster.

It appears that, in all cases, management and upgrade of the Karpenter release are not under the eksctl umbrella.

[CrisNevares]

+1

[AndrewFarley]

I would prefer to manage the karpenter Helm release myself, but the AWS tags, IAM resources, and groups make more sense from eksctl, and it would certainly be useful to be able to update these on an already-created eksctl cluster.

It appears that, in all cases, management and upgrade of the Karpenter release are not under the eksctl umbrella.

If this is the case, we should entirely remove all code in EKSCTL from setting up or managing Karpenter (on new clusters). We can't have it both ways, I'd argue. :) Support Karpenter everywhere and its' updates and maintenance via this tool, or remove it entirely. Thoughts?

[thurahtetaung]

I agree, this is a much needed feature. Currently, it feels like Karpenter support is there just for the sake of being there because not everyone installed it along with eksctl cluster creation, and many who didn't are stuck having to manually install and mess with eksctl managed resources manually if they want to use Karpenter. And Karpenter is getting quite popular so the demand for this feature will only grow. I think having this feature and being able to migrate from CAS to Karpenter just with eksctl commands will be very helpful.

[josegonzalez]

I think the annoying part here is that the Karpenter docs seem to only display how to install karpenter via eksctl, so if you're coming to Karpenter with an existing cluster, you're not in a great state.

Seems like maybe following the docs for migrating from CAS would be helpful, but its not marketed that way for new users, so again, not the most intuitive process to get it started.

[aogier]

I think the annoying part here is that the Karpenter docs seem to only display how to install karpenter via eksctl, so if you're coming to Karpenter with an existing cluster, you're not in a great state.

you literally linked helm install instructions, they look like a good starting point to me

[josegonzalez]

There's a bunch of stuff outside of the helm chart that needs to be provisioned - iam roles, access to the cluster via authconfig, etc. Yes, you can suss that out via the migration docs, but that wasn't clear to me at first glance.

[josegonzalez]

I'd like to add that it's clear to me from the Karpenter issue tracker that it's not always clear to others how to get things working:

• Using Access Entries for Karpenter isn't super clear (doesn't work without undocumented tricks): Update Getting Started Guide to Use EKS Access Entry API aws/karpenter-provider-aws#5369
• Someone asking for docs for setting up Karpenter on an existing cluster: Need clear installation documentation aws/karpenter-provider-aws#4722
• Asking for clearer docs, "Including for existing Clusters": Clearer Install Instructions, Including for existing EKS Clusters aws/karpenter-provider-aws#7452
• Clarifying the migration docs, which we are supposed to lean on: Clarify v1 migration aws/karpenter-provider-aws#6808
• Inconsistencies between the getting started and migration docs (which I personally discovered myself): Improve consistency of Getting Started and Migration guides aws/karpenter-provider-aws#6877

What I mean to say with this is that while I understand that adding this functionality to eksctl isn't "free" and requires effort, I don't think pointing at the Karpenter docs is a good alternative, and I think others would agree with me :)