Add support for wellKnownPolicies when defining addons

[aclevername]

What feature/behavior/change do you want?
When creating an IAMSerivceAccount you can reference wellKnownPolicies, e.g.:

iam:
  withOIDC: true
  serviceAccounts:
  - metadata:
      name: aws-load-balancer-controller
      namespace: kube-system
    wellKnownPolicies:
      awsLoadBalancerController: true

but you can't do this for addons, depsite it creating an equiavlent IAMServiceAccount under the hood.

Example

The following config file:

addons:
- name: some-new-addon
  wellKnownPolicies:
    awsLoadBalancerController: true

Should result in the policies for awsLoadBalancerController getting added to the addons IAMServiceAccount. It should be equivalent to:

addons:
- name: some-new-addon
  attachPolicy: # inline policy can be defined along with `attachPolicyARNs`
    Version: "2012-10-17"
    Statement:
    - Effect: Allow
      Action:
      - "awsLoadBalancerController-actions"
      ...
      Resource: '*'
      ....

Why do you want this feature?
As more addons get introduced some of the existing wellKnownPolicies will be applicable, being able to use this is convienant

[Callisto13]

Just to clarify, these wellKnownPolicies will be extra things that the user wants the addon to have access to, not policies that the addon actually needs to be able to work?

[aclevername]

Just to clarify, these wellKnownPolicies will be extra things that the user wants the addon to have access to, not policies that the addon actually needs to be able to work?

Well it can be either. For example when EBS CSI Driver addon gets introduced the user can select the well known policy for that #4188

[Callisto13]

why would we not just automatically attach the required policies? i know things like that are coming up, but why expose this?

[Skarlso]

The name, wellKnown was misleading. I literally thought it's an AWS policy feature I somehow missed. :)

Turns out it's just common policies people attach to clusters. I really would prefer this to be called commonPolicies or commonClusterPolicies or even mostUsedPolicies instead. :) But it's not a blocker.

Why? Because wellKnown is a connotation that brings baggage so to speak. It's a loaded phrase with a different historical meaning to it that made me think it's an embedded feature.

[aclevername]

why would we not just automatically attach the required policies? i know things like that are coming up, but why expose this?

I think theres a few reasons:

1. We try-our-best to make sure we automatically assign the recommended roles for an addon if a user doesn't specify it. But its on us to remember to update the code. This approach means if someone released an auto-scaler addon tomorrow, a user could easily get it working by using the wellKnownPolicies
2. Its more declarative to write:

addons:
- name: some-new-addon
  wellKnownPolicies:
    awsLoadBalancerController: true

than

addons:
- name: some-new-addon

where you sort of just "hope" eksctl knows to attach the correct addons

[aclevername]

The name, wellKnown was misleading. I literally thought it's an AWS policy feature I somehow missed. :)

Turns out it's just common policies people attach to clusters. I really would prefer this to be called commonPolicies or commonClusterPolicies or even mostUsedPolicies instead. :) But it's not a blocker.

Why? Because wellKnown is a connotation that brings baggage so to speak. It's a loaded phrase with a different historical meaning to it that made me think it's an embedded feature.

I don't think any of your suggestions are better or worse 😄 its a very hard thing to name. But the name is already in use in the IAM service accounts config section, and keeping naming parity with that makes sense, so I don't think changing the name just for the addons part makes sense.

[Callisto13]

Just to check: this is for managed addons, correct?

[aclevername]

Just to check: this is for managed addons, correct?

yes

[tanvp112]

How does this impact iam.serviceAccounts, do we still need to declare wellKnownPolicies for each service account?

Does the addon create the service account altogether? I hope not because sometimes we used different service account name other than the default.

[aclevername]

How does this impact iam.serviceAccounts, do we still need to declare wellKnownPolicies for each service account?

It doesn't affect IAM.ServiceAcounts at all

Does the addon create the service account altogether? I hope not because sometimes we used different service account name other than the default

For EKS addon it does not. Its the responsibility of the EKS addon to create the service account, we just pass it the role ARN.

[tanvp112]

@aclevername, just to be clear, despite that the addons already specified wellKnownPolicies, we still have to specify again in iam.serviceAccounts wellKnownPolicies field , doing so will not result to eksctl to load the policy twice? Example:

iam:
  serviceAccounts:
  - metadata:
      name: aws-load-balancer-controller
      namespace: kube-system
    wellKnownPolicies:
      awsLoadBalancerController: true
...
addons:
- name: awsLoadBalancer-addon
  wellKnownPolicies:
    awsLoadBalancerController: true