SSM (System Manager), default, and IRA (IAM Roles anywhere). Required IRA config (i.e. TrustAnchor, AnywhereProfile) will be created by eksctl behind the scenes."
+ },
+ "roleARN": {
+ "type": "string",
+ "description": "the IAM Role ARN to be added to aws-auth configmap for remote nodes. If not set, eksctl creates the role behind the scenes, adds an entry into the configmap and sets up any other SSM/IRA config. If set, eksctl will only add the configmap entry, while creating any required SSM/IRA config falls under user's responsibility.",
+ "x-intellij-html-description": "the IAM Role ARN to be added to aws-auth configmap for remote nodes. If not set, eksctl creates the role behind the scenes, adds an entry into the configmap and sets up any other SSM/IRA config. If set, eksctl will only add the configmap entry, while creating any required SSM/IRA config falls under user's responsibility."
+ }
+ },
+ "preferredOrder": [
+ "provider",
+ "roleARN",
+ "caBundleCert"
+ ],
+ "additionalProperties": false
+ },
"SecretsEncryption": {
"required": [
"keyARN"
@@ -2522,6 +2602,11 @@
"description": "defines the configuration for KMS encryption provider",
"x-intellij-html-description": "defines the configuration for KMS encryption provider"
},
+ "VPCGateway": {
+ "type": "string",
+ "description": "VPCGatewayID the ID of the gateway that facilitates external connectivity from customer's VPC to their remote network(s). Valid options are Transit Gateway and Virtual Private Gateway.",
+ "x-intellij-html-description": "VPCGatewayID the ID of the gateway that facilitates external connectivity from customer's VPC to their remote network(s). Valid options are Transit Gateway and Virtual Private Gateway."
+ },
"VolumeMapping": {
"properties": {
"snapshotID": {
diff --git a/pkg/apis/eksctl.io/v1alpha5/defaults.go b/pkg/apis/eksctl.io/v1alpha5/defaults.go
index a7574538b1..08fb81ba86 100644
--- a/pkg/apis/eksctl.io/v1alpha5/defaults.go
+++ b/pkg/apis/eksctl.io/v1alpha5/defaults.go
@@ -79,6 +79,15 @@ func SetClusterConfigDefaults(cfg *ClusterConfig) {
if cfg.Karpenter != nil && cfg.Karpenter.CreateServiceAccount == nil {
cfg.Karpenter.CreateServiceAccount = Disabled()
}
+
+ if cfg.RemoteNetworkConfig != nil {
+ if cfg.RemoteNetworkConfig.IAM == nil {
+ cfg.RemoteNetworkConfig.IAM = &RemoteNodesIAM{}
+ }
+ if cfg.RemoteNetworkConfig.IAM.Provider == nil {
+ cfg.RemoteNetworkConfig.IAM.Provider = &SSMProvider
+ }
+ }
}
// IAMServiceAccountsWithImplicitServiceAccounts adds implicitly created
diff --git a/pkg/apis/eksctl.io/v1alpha5/defaults_test.go b/pkg/apis/eksctl.io/v1alpha5/defaults_test.go
index e47147e0dc..b015b0a39e 100644
--- a/pkg/apis/eksctl.io/v1alpha5/defaults_test.go
+++ b/pkg/apis/eksctl.io/v1alpha5/defaults_test.go
@@ -370,6 +370,14 @@ var _ = Describe("ClusterConfig validation", func() {
cfg = NewClusterConfig()
})
+ Describe("RemoteNetworkConfig", func() {
+ It("should set default credentials provider to SSM", func() {
+ cfg.RemoteNetworkConfig = &RemoteNetworkConfig{}
+ SetClusterConfigDefaults(cfg)
+ Expect(*cfg.RemoteNetworkConfig.IAM.Provider).To(Equal(SSMProvider))
+ })
+ })
+
Describe("SetDefaultFargateProfile", func() {
It("should create a default Fargate profile with two selectors matching default and kube-system w/o any label", func() {
Expect(cfg.FargateProfiles).To(HaveLen(0))
diff --git a/pkg/apis/eksctl.io/v1alpha5/partitions.go b/pkg/apis/eksctl.io/v1alpha5/partitions.go
index 1bc9b82e46..d074e1a9a9 100644
--- a/pkg/apis/eksctl.io/v1alpha5/partitions.go
+++ b/pkg/apis/eksctl.io/v1alpha5/partitions.go
@@ -24,6 +24,8 @@ type partitions []partition
var standardServiceMappings = map[string]string{
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
+ "SSM": "ssm.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
"EKSFargatePods": "eks-fargate-pods.amazonaws.com",
}
diff --git a/pkg/apis/eksctl.io/v1alpha5/types.go b/pkg/apis/eksctl.io/v1alpha5/types.go
index 867b8b5d7e..758d3907e0 100644
--- a/pkg/apis/eksctl.io/v1alpha5/types.go
+++ b/pkg/apis/eksctl.io/v1alpha5/types.go
@@ -722,6 +722,83 @@ func (k *KubernetesNetworkConfig) IPv6Enabled() bool {
return strings.EqualFold(k.IPFamily, IPV6Family)
}
+// VPCGatewayID the ID of the gateway that facilitates external connectivity
+// from customer's VPC to their remote network(s).
+// Valid options are Transit Gateway and Virtual Private Gateway.
+type VPCGateway string
+
+var (
+ SSMProvider = "ssm"
+ IRAProvider = "ira"
+ transitGatewayPrefix = "tgw"
+ virtualPrivateGatewayPrefix = "vgw"
+)
+
+func (v *VPCGateway) IsSet() bool {
+ return v != nil && *v != ""
+}
+
+func (v *VPCGateway) IsTransitGateway() bool {
+ return v != nil && strings.HasPrefix(string(*v), transitGatewayPrefix)
+}
+
+func (v *VPCGateway) IsVirtualPrivateGateway() bool {
+ return v != nil && strings.HasPrefix(string(*v), virtualPrivateGatewayPrefix)
+}
+
+// RemoteNetworkConfig
+type RemoteNetworkConfig struct {
+ // +optional
+ IAM *RemoteNodesIAM `json:"iam,omitempty"`
+ // +required
+ VPCGatewayID *VPCGateway `json:"vpcGatewayID,omitempty"`
+ // +required
+ RemoteNodeNetworks []*RemoteNetwork `json:"remoteNodeNetworks,omitempty"`
+ // +optional
+ RemotePodNetworks []*RemoteNetwork `json:"remotePodNetworks,omitempty"`
+}
+
+type RemoteNodesIAM struct {
+ // Provider the AWS service responsible for provisioning IAM credentials to remote nodes.
+ // Valid options are `SSM` (System Manager), default, and `IRA` (IAM Roles anywhere).
+ // Required IRA config (i.e. TrustAnchor, AnywhereProfile) will be created by eksctl behind the scenes.
+ // +optional
+ Provider *string `json:"provider,omitempty"`
+ // RoleARN the IAM Role ARN to be added to aws-auth configmap for remote nodes.
+ // If not set, eksctl creates the role behind the scenes, adds an entry into the configmap and sets up any other SSM/IRA config.
+ // If set, eksctl will only add the configmap entry, while creating any required SSM/IRA config falls under user's responsibility.
+ // +optional
+ RoleARN *string `json:"roleARN,omitempty"`
+ // CABundleCert the CA bundle certificate used by IRA trust anchor.
+ // Can't be set if Provider is SSM.
+ // +optional
+ CABundleCert *string `json:"caBundleCert,omitempty"`
+}
+
+// RemoteNetwork
+type RemoteNetwork struct {
+ CIDRs []string `json:"cidrs,omitempty"`
+}
+
+func (r *RemoteNetworkConfig) ToRemoteNetworksPool() []string {
+ remoteNetworksPool := []string{}
+ for _, r := range r.RemoteNodeNetworks {
+ remoteNetworksPool = append(remoteNetworksPool, r.CIDRs...)
+ }
+ for _, r := range r.RemotePodNetworks {
+ remoteNetworksPool = append(remoteNetworksPool, r.CIDRs...)
+ }
+ return remoteNetworksPool
+}
+
+func (r *RemoteNetworkConfig) HasRemoteNodesEnabled() bool {
+ return r.RemoteNodeNetworks != nil && len(r.RemoteNodeNetworks) > 0
+}
+
+func (c *ClusterConfig) HasRemoteNetworkingConfigured() bool {
+ return c.RemoteNetworkConfig != nil && (len(c.RemoteNetworkConfig.RemoteNodeNetworks) > 0 || len(c.RemoteNetworkConfig.RemotePodNetworks) > 0)
+}
+
type EKSCTLCreated string
// ClusterStatus holds read-only attributes of a cluster
@@ -898,6 +975,9 @@ type ClusterConfig struct {
// +optional
AutoModeConfig *AutoModeConfig `json:"autoModeConfig,omitempty"`
+ // +optional
+ RemoteNetworkConfig *RemoteNetworkConfig `json:"remoteNetworkConfig,omitempty"`
+
// +optional
IAM *ClusterIAM `json:"iam,omitempty"`
diff --git a/pkg/apis/eksctl.io/v1alpha5/validation.go b/pkg/apis/eksctl.io/v1alpha5/validation.go
index 1875aeca79..745380ede9 100644
--- a/pkg/apis/eksctl.io/v1alpha5/validation.go
+++ b/pkg/apis/eksctl.io/v1alpha5/validation.go
@@ -82,6 +82,71 @@ func setNonEmpty(field string) error {
return fmt.Errorf("%s must be set and non-empty", field)
}
+func (c *ClusterConfig) validateRemoteNetworkingConfig() error {
+ rnc := c.RemoteNetworkConfig
+ if rnc == nil {
+ return nil
+ }
+
+ if !IsEnabled(c.VPC.ClusterEndpoints.PublicAccess) {
+ return fmt.Errorf("remoteNetworkConfig requires public cluster endpoint access")
+ }
+
+ if c.IsFullyPrivate() {
+ return fmt.Errorf("remoteNetworkConfig is not supported on fully private EKS cluster")
+ }
+
+ if c.IPv6Enabled() {
+ return fmt.Errorf("remoteNetworkConfig is not supported on EKS cluster configured with IPv6 address family")
+ }
+
+ if c.AccessConfig.AuthenticationMode == ekstypes.AuthenticationModeConfigMap {
+ return fmt.Errorf("remoteNetworkConfig requires authenticationMode to be either %q or %q", ekstypes.AuthenticationModeApiAndConfigMap, ekstypes.AuthenticationModeApi)
+ }
+
+ if len(rnc.RemoteNodeNetworks) == 0 {
+ return setNonEmpty("remoteNetworkConfig.remoteNodeNetworks")
+ }
+
+ if c.VPC.ID != "" {
+ if rnc.VPCGatewayID.IsSet() {
+ return fmt.Errorf("remoteNetworkConfig.vpcGatewayID is not supported when using pre-existing VPC")
+ }
+ } else {
+ if !rnc.VPCGatewayID.IsSet() {
+ return setNonEmpty("remoteNetworkConfig.vpcGatewayID")
+ }
+ // vpcGatewayId must be either a virtual private gateway or a transit gateway
+ if !rnc.VPCGatewayID.IsTransitGateway() && !rnc.VPCGatewayID.IsVirtualPrivateGateway() {
+ return fmt.Errorf("invalid value %q provided for remoteNetworkConfig.vpcGatewayID; "+
+ "only transit gateway (tgw-*) or virtual private gateway (vgw-*) IDs are supported", *rnc.VPCGatewayID)
+ }
+ }
+
+ // credentials provider must be either SSM or IAM Roles Anywhere
+ if !strings.EqualFold(*rnc.IAM.Provider, SSMProvider) &&
+ !strings.EqualFold(*rnc.IAM.Provider, IRAProvider) {
+ return fmt.Errorf("invalid value %q provided for remoteNetworkConfig.iam.provider; only %q and %q are supported",
+ *rnc.IAM.Provider, SSMProvider, IRAProvider)
+ }
+
+ // CABundleCert should only be set of credentials provider is IAM Roles Anywhere
+ if strings.EqualFold(*rnc.IAM.Provider, SSMProvider) && rnc.IAM.CABundleCert != nil {
+ return fmt.Errorf("remoteNetworkConfig.iam.caBundleCert is not supported when using SSM credentials provider")
+ }
+
+ if strings.EqualFold(*rnc.IAM.Provider, IRAProvider) && rnc.IAM.CABundleCert == nil {
+ return fmt.Errorf("remoteNetworkConfig.iam.caBundleCert is required when using IAMRolesAnywhere credentials provider")
+ }
+
+ if IsSetAndNonEmptyString(rnc.IAM.RoleARN) {
+ logger.Warning("remoteNetworkConfig.iam.roleARN is set; eksctl will add a corresponding entry in aws-auth configmap; " +
+ "but won't setup an additional SSM or IAMRolesAnywhere required config")
+ }
+
+ return nil
+}
+
// ValidateClusterConfig checks compatible fields of a given ClusterConfig
func ValidateClusterConfig(cfg *ClusterConfig) error {
if IsDisabled(cfg.IAM.WithOIDC) && len(cfg.IAM.ServiceAccounts) > 0 {
@@ -106,6 +171,10 @@ func ValidateClusterConfig(cfg *ClusterConfig) error {
return err
}
+ if err := cfg.validateRemoteNetworkingConfig(); err != nil {
+ return err
+ }
+
// names must be unique across both managed and unmanaged nodegroups
ngNames := nameSet{}
validateNg := func(ng *NodeGroupBase, path string) error {
diff --git a/pkg/apis/eksctl.io/v1alpha5/validation_test.go b/pkg/apis/eksctl.io/v1alpha5/validation_test.go
index 1fa4b2eed1..5a104bb11a 100644
--- a/pkg/apis/eksctl.io/v1alpha5/validation_test.go
+++ b/pkg/apis/eksctl.io/v1alpha5/validation_test.go
@@ -5,6 +5,7 @@ import (
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
+ ekstypes "github.com/aws/aws-sdk-go-v2/service/eks/types"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
@@ -945,6 +946,104 @@ var _ = Describe("ClusterConfig validation", func() {
})
})
})
+
+ type remoteNetworkConfigEntry struct {
+ overrideConfig func(*api.ClusterConfig)
+ expectedErr string
+ }
+ DescribeTable("RemoteNetworkConfig", func(e remoteNetworkConfigEntry) {
+ cfg := api.NewClusterConfig()
+ api.SetClusterConfigDefaults(cfg)
+ api.SetClusterEndpointAccessDefaults(cfg.VPC)
+ gatewayID := api.VPCGateway("tgw-1234")
+ cfg.RemoteNetworkConfig = &api.RemoteNetworkConfig{
+ RemoteNodeNetworks: []*api.RemoteNetwork{{CIDRs: []string{"192.168.0.1"}}},
+ VPCGatewayID: &gatewayID,
+ }
+ e.overrideConfig(cfg)
+ Expect(api.ValidateClusterConfig(cfg)).To(MatchError(ContainSubstring(e.expectedErr)))
+ },
+
+ Entry("public cluster endpoint access is disabled", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.VPC.ClusterEndpoints = &api.ClusterEndpoints{
+ PublicAccess: api.Disabled(),
+ }
+ },
+ expectedErr: "remoteNetworkConfig requires public cluster endpoint access",
+ }),
+ Entry("fully private EKS cluster", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.PrivateCluster = &api.PrivateCluster{
+ Enabled: true,
+ }
+ },
+ expectedErr: "remoteNetworkConfig is not supported on fully private EKS cluster",
+ }),
+ Entry("IPv6 family", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.KubernetesNetworkConfig.IPFamily = api.IPV6Family
+ // setup other pre-requisites for IPv6
+ cc.IAM.WithOIDC = aws.Bool(true)
+ cc.Addons = []*api.Addon{
+ {Name: api.VPCCNIAddon},
+ {Name: api.CoreDNSAddon},
+ {Name: api.KubeProxyAddon},
+ }
+ },
+ expectedErr: "remoteNetworkConfig is not supported on EKS cluster configured with IPv6 address family",
+ }),
+ Entry("authenticationMode is ConfigMap", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.AccessConfig = &api.AccessConfig{
+ AuthenticationMode: ekstypes.AuthenticationModeConfigMap,
+ }
+ },
+ expectedErr: fmt.Sprintf("remoteNetworkConfig requires authenticationMode to be either %q or %q", ekstypes.AuthenticationModeApiAndConfigMap, ekstypes.AuthenticationModeApi),
+ }),
+ Entry("remoteNodeNetworks is empty", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.RemoteNetworkConfig = &api.RemoteNetworkConfig{}
+ },
+ expectedErr: "remoteNetworkConfig.remoteNodeNetworks must be set and non-empty",
+ }),
+ Entry("both vpcGatewayID and pre-existing VPC are set", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.VPC.ID = "vpc-1234"
+ },
+ expectedErr: "remoteNetworkConfig.vpcGatewayID is not supported when using pre-existing VPC",
+ }),
+ Entry("both vpcGatewayID and pre-existing VPC are missing", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.RemoteNetworkConfig.VPCGatewayID = nil
+ },
+ expectedErr: "remoteNetworkConfig.vpcGatewayID must be set and non-empty",
+ }),
+ Entry("unsupported vpcGateway type", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ gatewayID := api.VPCGateway("igw-1234")
+ cc.RemoteNetworkConfig.VPCGatewayID = &gatewayID
+ },
+ expectedErr: "invalid value \"igw-1234\" provided for remoteNetworkConfig.vpcGatewayID",
+ }),
+ Entry("unsupported credentials provider", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.RemoteNetworkConfig.IAM = &api.RemoteNodesIAM{
+ Provider: aws.String("BLOB"),
+ }
+ },
+ expectedErr: "invalid value \"BLOB\" provided for remoteNetworkConfig.iam.provider",
+ }),
+ Entry("CABundle cert is missing when using IRA", remoteNetworkConfigEntry{
+ overrideConfig: func(cc *api.ClusterConfig) {
+ cc.RemoteNetworkConfig.IAM = &api.RemoteNodesIAM{
+ Provider: aws.String("IRA"),
+ }
+ },
+ expectedErr: "remoteNetworkConfig.iam.caBundleCert is required when using IAMRolesAnywhere credentials provider",
+ }),
+ )
+
Describe("network config", func() {
var (
cfg *api.ClusterConfig
diff --git a/pkg/apis/eksctl.io/v1alpha5/zz_generated.deepcopy.go b/pkg/apis/eksctl.io/v1alpha5/zz_generated.deepcopy.go
index 0574339a83..adecb5075b 100644
--- a/pkg/apis/eksctl.io/v1alpha5/zz_generated.deepcopy.go
+++ b/pkg/apis/eksctl.io/v1alpha5/zz_generated.deepcopy.go
@@ -394,6 +394,11 @@ func (in *ClusterConfig) DeepCopyInto(out *ClusterConfig) {
*out = new(KubernetesNetworkConfig)
**out = **in
}
+ if in.RemoteNetworkConfig != nil {
+ in, out := &in.RemoteNetworkConfig, &out.RemoteNetworkConfig
+ *out = new(RemoteNetworkConfig)
+ (*in).DeepCopyInto(*out)
+ }
if in.AutoModeConfig != nil {
in, out := &in.AutoModeConfig, &out.AutoModeConfig
*out = new(AutoModeConfig)
@@ -2127,6 +2132,106 @@ func (in *ProviderConfig) DeepCopy() *ProviderConfig {
return out
}
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *RemoteNetwork) DeepCopyInto(out *RemoteNetwork) {
+ *out = *in
+ if in.CIDRs != nil {
+ in, out := &in.CIDRs, &out.CIDRs
+ *out = make([]string, len(*in))
+ copy(*out, *in)
+ }
+ return
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteNetwork.
+func (in *RemoteNetwork) DeepCopy() *RemoteNetwork {
+ if in == nil {
+ return nil
+ }
+ out := new(RemoteNetwork)
+ in.DeepCopyInto(out)
+ return out
+}
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *RemoteNetworkConfig) DeepCopyInto(out *RemoteNetworkConfig) {
+ *out = *in
+ if in.IAM != nil {
+ in, out := &in.IAM, &out.IAM
+ *out = new(RemoteNodesIAM)
+ (*in).DeepCopyInto(*out)
+ }
+ if in.VPCGatewayID != nil {
+ in, out := &in.VPCGatewayID, &out.VPCGatewayID
+ *out = new(VPCGateway)
+ **out = **in
+ }
+ if in.RemoteNodeNetworks != nil {
+ in, out := &in.RemoteNodeNetworks, &out.RemoteNodeNetworks
+ *out = make([]*RemoteNetwork, len(*in))
+ for i := range *in {
+ if (*in)[i] != nil {
+ in, out := &(*in)[i], &(*out)[i]
+ *out = new(RemoteNetwork)
+ (*in).DeepCopyInto(*out)
+ }
+ }
+ }
+ if in.RemotePodNetworks != nil {
+ in, out := &in.RemotePodNetworks, &out.RemotePodNetworks
+ *out = make([]*RemoteNetwork, len(*in))
+ for i := range *in {
+ if (*in)[i] != nil {
+ in, out := &(*in)[i], &(*out)[i]
+ *out = new(RemoteNetwork)
+ (*in).DeepCopyInto(*out)
+ }
+ }
+ }
+ return
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteNetworkConfig.
+func (in *RemoteNetworkConfig) DeepCopy() *RemoteNetworkConfig {
+ if in == nil {
+ return nil
+ }
+ out := new(RemoteNetworkConfig)
+ in.DeepCopyInto(out)
+ return out
+}
+
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *RemoteNodesIAM) DeepCopyInto(out *RemoteNodesIAM) {
+ *out = *in
+ if in.Provider != nil {
+ in, out := &in.Provider, &out.Provider
+ *out = new(string)
+ **out = **in
+ }
+ if in.RoleARN != nil {
+ in, out := &in.RoleARN, &out.RoleARN
+ *out = new(string)
+ **out = **in
+ }
+ if in.CABundleCert != nil {
+ in, out := &in.CABundleCert, &out.CABundleCert
+ *out = new(string)
+ **out = **in
+ }
+ return
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteNodesIAM.
+func (in *RemoteNodesIAM) DeepCopy() *RemoteNodesIAM {
+ if in == nil {
+ return nil
+ }
+ out := new(RemoteNodesIAM)
+ in.DeepCopyInto(out)
+ return out
+}
+
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *ScalingConfig) DeepCopyInto(out *ScalingConfig) {
*out = *in
diff --git a/pkg/cfn/builder/cluster.go b/pkg/cfn/builder/cluster.go
index 89a670ffc9..0212bd2b0a 100644
--- a/pkg/cfn/builder/cluster.go
+++ b/pkg/cfn/builder/cluster.go
@@ -89,6 +89,9 @@ func (c *ClusterResourceSet) AddAllResources(ctx context.Context) error {
if err := c.addResourcesForControlPlane(subnetDetails); err != nil {
return err
}
+ if c.spec.HasRemoteNetworkingConfigured() {
+ c.addAccessEntryForRemoteNodes()
+ }
if len(c.spec.FargateProfiles) > 0 {
c.addResourcesForFargate()
@@ -130,6 +133,19 @@ func (c *ClusterResourceSet) addResourcesForSecurityGroups(vpcID *gfnt.Value) *c
VpcId: vpcID,
})
+ if c.spec.HasRemoteNetworkingConfigured() {
+ for i, remoteNetworkCIRD := range c.spec.RemoteNetworkConfig.ToRemoteNetworksPool() {
+ c.newResource(fmt.Sprintf("IngressControlPlaneRemoteNetworks%d", i), &gfnec2.SecurityGroupIngress{
+ GroupId: refControlPlaneSG,
+ CidrIp: gfnt.NewString(remoteNetworkCIRD),
+ Description: gfnt.NewString(fmt.Sprintf("Allow nodes/pods from remote network (%s) to communicate to controlplane", remoteNetworkCIRD)),
+ IpProtocol: gfnt.NewString("tcp"),
+ FromPort: sgPortHTTPS,
+ ToPort: sgPortHTTPS,
+ })
+ }
+ }
+
if len(c.spec.VPC.ExtraCIDRs) > 0 {
for i, cidr := range c.spec.VPC.ExtraCIDRs {
c.newResource(fmt.Sprintf("IngressControlPlaneExtraCIDR%d", i), &gfnec2.SecurityGroupIngress{
@@ -378,6 +394,20 @@ func (c *ClusterResourceSet) addResourcesForControlPlane(subnetDetails *SubnetDe
}
}
+ if c.spec.HasRemoteNetworkingConfigured() {
+ cluster.RemoteNetworkConfig = &gfneks.Cluster_RemoteNetworkConfig{}
+ for _, remoteNetwork := range c.spec.RemoteNetworkConfig.RemotePodNetworks {
+ cluster.RemoteNetworkConfig.RemotePodNetworks = append(cluster.RemoteNetworkConfig.RemotePodNetworks, gfneks.RemoteNetworks{
+ CIDRs: gfnt.NewStringSlice(remoteNetwork.CIDRs...),
+ })
+ }
+ for _, remoteNetwork := range c.spec.RemoteNetworkConfig.RemoteNodeNetworks {
+ cluster.RemoteNetworkConfig.RemoteNodeNetworks = append(cluster.RemoteNetworkConfig.RemoteNodeNetworks, gfneks.RemoteNetworks{
+ CIDRs: gfnt.NewStringSlice(remoteNetwork.CIDRs...),
+ })
+ }
+ }
+
c.newResource("ControlPlane", &cluster)
if c.spec.Status == nil {
@@ -412,6 +442,26 @@ func (c *ClusterResourceSet) addResourcesForControlPlane(subnetDetails *SubnetDe
return nil
}
+func (c *ClusterResourceSet) addAccessEntryForRemoteNodes() {
+ getRemoteNodesRoleName := func() string {
+ switch *c.spec.RemoteNetworkConfig.IAM.Provider {
+ case api.SSMProvider:
+ return SSMRole
+ case api.IRAProvider:
+ return IRARole
+ default:
+ // Validations should ensure this is never reached
+ return ""
+ }
+ }
+ c.newResource("RemoteNodesAccessEntry", &gfneks.AccessEntry{
+ PrincipalArn: gfnt.MakeFnGetAttString(getRemoteNodesRoleName(), "Arn"),
+ ClusterName: gfnt.NewString(c.spec.Metadata.Name),
+ Type: gfnt.NewString(string(api.AccessEntryTypeHybridLinux)),
+ AWSCloudFormationDependsOn: []string{"ControlPlane"},
+ })
+}
+
func makeCFNTags(clusterConfig *api.ClusterConfig) []gfncfn.Tag {
var tags []gfncfn.Tag
for k, v := range clusterConfig.Metadata.Tags {
diff --git a/pkg/cfn/builder/cluster_test.go b/pkg/cfn/builder/cluster_test.go
index ca5bd24083..97a5833d31 100644
--- a/pkg/cfn/builder/cluster_test.go
+++ b/pkg/cfn/builder/cluster_test.go
@@ -633,6 +633,66 @@ var _ = Describe("Cluster Template Builder", func() {
})
})
+ Context("when RemoteNetworkConfig is set", func() {
+ var (
+ gatewayID api.VPCGateway
+ )
+ BeforeEach(func() {
+ gatewayID = api.VPCGateway("tgw-1234")
+ cfg.RemoteNetworkConfig = &api.RemoteNetworkConfig{
+ IAM: &api.RemoteNodesIAM{
+ Provider: &api.IRAProvider,
+ },
+ VPCGatewayID: &gatewayID,
+ RemoteNodeNetworks: []*api.RemoteNetwork{{CIDRs: []string{"192.168.0.1"}}},
+ }
+ })
+ It("should create all appropriate resources", func() {
+ // should create all networking resources for the VPC
+ Expect(clusterTemplate.Resources).To(HaveKey("TransitGatewayAttachment"))
+ tgwAttachment := clusterTemplate.Resources["TransitGatewayAttachment"]
+ Expect(tgwAttachment.Properties.TransitGatewayId).To(Equal("tgw-1234"))
+ Expect(tgwAttachment.Properties.TransitGatewayId).To(Equal("tgw-1234"))
+
+ Expect(clusterTemplate.Resources).To(HaveKey("TGWPrivateSubnetRoute0USWEST2A"))
+ tgwRoute := clusterTemplate.Resources["TGWPrivateSubnetRoute0USWEST2A"]
+ Expect(tgwRoute.Properties.TransitGatewayId).To(Equal("tgw-1234"))
+ Expect(tgwRoute.Properties.DestinationCidrBlock).To(Equal("192.168.0.1"))
+
+ Expect(clusterTemplate.Resources).To(HaveKey("TGWPrivateSubnetRoute0USWEST2B"))
+ tgwRoute = clusterTemplate.Resources["TGWPrivateSubnetRoute0USWEST2B"]
+ Expect(tgwRoute.Properties.TransitGatewayId).To(Equal("tgw-1234"))
+ Expect(tgwRoute.Properties.DestinationCidrBlock).To(Equal("192.168.0.1"))
+
+ Expect(clusterTemplate.Resources).To(HaveKey("IngressControlPlaneRemoteNetworks0"))
+ ingressRemote := clusterTemplate.Resources["IngressControlPlaneRemoteNetworks0"]
+ Expect(ingressRemote.Properties.CidrIP).To(Equal("192.168.0.1"))
+ Expect(ingressRemote.Properties.FromPort).To(Equal(443))
+ Expect(ingressRemote.Properties.ToPort).To(Equal(443))
+
+ // should create IAM Roles Anywhere resources
+ Expect(clusterTemplate.Resources).To(HaveKey("TrustAnchor"))
+
+ Expect(clusterTemplate.Resources).To(HaveKey("AnywhereProfile"))
+ anywhereProfile := clusterTemplate.Resources["AnywhereProfile"]
+ Expect(anywhereProfile.Properties.AcceptRoleSessionName).To(BeTrue())
+
+ Expect(clusterTemplate.Resources).To(HaveKey("HybridNodesIRARole"))
+ iraRole := clusterTemplate.Resources["HybridNodesIRARole"]
+ Expect(iraRole.Properties.ManagedPolicyArns).To(ContainElement(map[string]interface{}{
+ "Fn::Sub": "arn:${AWS::Partition}:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly",
+ }))
+ Expect(iraRole.Properties.ManagedPolicyArns).To(ContainElement(map[string]interface{}{
+ "Fn::Sub": "arn:${AWS::Partition}:iam::aws:policy/AmazonSSMManagedInstanceCore",
+ }))
+
+ // should create a HYBRID_LINUX access entry to allow remote notes to join the cluster
+ Expect(clusterTemplate.Resources).To(HaveKey("RemoteNodesAccessEntry"))
+ remoteAE := clusterTemplate.Resources["RemoteNodesAccessEntry"]
+ Expect(remoteAE.Properties.Type).To(Equal("HYBRID_LINUX"))
+ })
+ })
+
Context("when adding vpc resources fails", func() {
BeforeEach(func() {
cfg.VPC = &api.ClusterVPC{}
diff --git a/pkg/cfn/builder/fakes/fake_cfn_template.go b/pkg/cfn/builder/fakes/fake_cfn_template.go
index 5e554f6d7e..1ff4a6c49e 100644
--- a/pkg/cfn/builder/fakes/fake_cfn_template.go
+++ b/pkg/cfn/builder/fakes/fake_cfn_template.go
@@ -20,6 +20,7 @@ type Tag struct {
}
type Properties struct {
+ AcceptRoleSessionName bool
EnableDNSHostnames, EnableDNSSupport bool
GroupDescription string
Description string
@@ -30,6 +31,7 @@ type Properties struct {
SourceSecurityGroupID interface{}
DestinationSecurityGroupID interface{}
+ Type string
Path, RoleName string
Roles, ManagedPolicyArns []interface{}
PermissionsBoundary interface{}
@@ -62,12 +64,12 @@ type Properties struct {
CidrIP, CidrIPv6, IPProtocol string
FromPort, ToPort int
- VpcID, SubnetID interface{}
- EgressOnlyInternetGatewayID, RouteTableID, AllocationID interface{}
- GatewayID, InternetGatewayID, NatGatewayID interface{}
- DestinationCidrBlock, DestinationIpv6CidrBlock interface{}
- MapPublicIPOnLaunch bool
- AssignIpv6AddressOnCreation *bool
+ VpcID, SubnetID interface{}
+ EgressOnlyInternetGatewayID, RouteTableID, AllocationID interface{}
+ GatewayID, InternetGatewayID, NatGatewayID, VpnGatewayId, TransitGatewayId interface{}
+ DestinationCidrBlock, DestinationIpv6CidrBlock interface{}
+ MapPublicIPOnLaunch bool
+ AssignIpv6AddressOnCreation *bool
Ipv6CidrBlock interface{}
Ipv6Pool string
@@ -77,7 +79,8 @@ type Properties struct {
AmazonProvidedIpv6CidrBlock bool
AvailabilityZone, Domain string
- Name, Version string
+ Name interface{}
+ Version string
RoleArn interface{}
ResourcesVpcConfig struct {
SecurityGroupIDs []interface{}
diff --git a/pkg/cfn/builder/iam.go b/pkg/cfn/builder/iam.go
index 74b6b59dd5..41f5ceb8f7 100644
--- a/pkg/cfn/builder/iam.go
+++ b/pkg/cfn/builder/iam.go
@@ -4,11 +4,12 @@ import (
"context"
"fmt"
+ "github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/cloudformation/types"
-
"github.com/kris-nova/logger"
gfniam "github.com/weaveworks/goformation/v4/cloudformation/iam"
+ gfnrolesanywhere "github.com/weaveworks/goformation/v4/cloudformation/rolesanywhere"
gfnt "github.com/weaveworks/goformation/v4/cloudformation/types"
api "github.com/weaveworks/eksctl/pkg/apis/eksctl.io/v1alpha5"
@@ -38,10 +39,39 @@ const (
cfnIAMInstanceProfileName = "NodeInstanceProfile"
)
+const (
+ TrustAnchor = "TrustAnchor"
+ AnywhereProfile = "AnywhereProfile"
+ IntermediateRole = "IntermediateRole"
+ IRARole = "HybridNodesIRARole"
+ SSMRole = "HybridNodesSSMRole"
+)
+
var (
iamDefaultNodePolicies = []string{
iamPolicyAmazonEKSWorkerNodePolicy,
}
+ eksDescribeClusterPolicy = gfniam.Role_Policy{
+ PolicyName: gfnt.NewString("EKSDescribeCluster"),
+ PolicyDocument: cft.MakePolicyDocument(cft.MapOfInterfaces{
+ "Effect": "Allow",
+ "Action": []string{
+ "eks:DescribeCluster",
+ },
+ "Resource": "*",
+ }),
+ }
+ ssmRolePolicy = gfniam.Role_Policy{
+ PolicyName: gfnt.NewString("SSMRolePolicy"),
+ PolicyDocument: cft.MakePolicyDocument(cft.MapOfInterfaces{
+ "Effect": "Allow",
+ "Action": []string{
+ "ssm:DeregisterManagedInstance",
+ "ssm:DescribeInstanceInformation",
+ },
+ "Resource": "*",
+ }),
+ }
)
func (c *resourceSet) attachAllowPolicy(name string, refRole *gfnt.Value, statements []cft.MapOfInterfaces) {
@@ -70,15 +100,7 @@ func (c *ClusterResourceSet) WithNamedIAM() bool {
return c.rs.withNamedIAM
}
-func (c *ClusterResourceSet) addResourcesForIAM() {
- c.rs.withNamedIAM = false
-
- if api.IsSetAndNonEmptyString(c.spec.IAM.ServiceRoleARN) {
- c.rs.withIAM = false
- c.rs.defineOutputWithoutCollector(outputs.ClusterServiceRoleARN, c.spec.IAM.ServiceRoleARN, true)
- return
- }
-
+func (c *ClusterResourceSet) addResourcesForServiceRole() {
c.rs.withIAM = true
var role *gfniam.Role
@@ -117,6 +139,117 @@ func (c *ClusterResourceSet) addResourcesForIAM() {
})
}
+func (c *ClusterResourceSet) addIAMRolesAnywhere() {
+ trustAnchor := &gfnrolesanywhere.TrustAnchor{
+ Enabled: gfnt.NewBoolean(true),
+ Name: makeName("CA"),
+ Source: &gfnrolesanywhere.TrustAnchor_Source{
+ SourceType: aws.String("CERTIFICATE_BUNDLE"),
+ SourceData: &gfnrolesanywhere.TrustAnchor_SourceData{
+ X509CertificateData: c.spec.RemoteNetworkConfig.IAM.CABundleCert,
+ },
+ },
+ }
+ anywhereProfile := &gfnrolesanywhere.Profile{
+ Enabled: gfnt.NewBoolean(true),
+ Name: makeName("remote-nodes"),
+ RoleArns: gfnt.NewSlice(
+ gfnt.MakeFnGetAttString(IRARole, "Arn"),
+ ),
+ AcceptRoleSessionName: gfnt.NewBoolean(true),
+ AWSCloudFormationDependsOn: []string{IRARole},
+ }
+ iraRole := &gfniam.Role{
+ AssumeRolePolicyDocument: cft.MakeAssumeRolePolicyDocumentForServicesWithConditionsAndActions(
+ cft.MapOfInterfaces{
+ "ArnEquals": cft.MapOfInterfaces{
+ "aws:SourceArn": gfnt.MakeFnGetAttString(TrustAnchor, "TrustAnchorArn"),
+ },
+ },
+ []string{
+ "sts:TagSession",
+ "sts:SetSourceIdentity",
+ },
+ MakeServiceRef("IRA"),
+ ),
+ Policies: []gfniam.Role_Policy{
+ eksDescribeClusterPolicy,
+ },
+ ManagedPolicyArns: gfnt.NewSlice(makePolicyARNs([]string{
+ iamPolicyAmazonEC2ContainerRegistryReadOnly,
+ iamPolicyAmazonSSMManagedInstanceCore,
+ }...)...),
+ AWSCloudFormationDependsOn: []string{TrustAnchor},
+ }
+
+ c.newResource(TrustAnchor, trustAnchor)
+ c.newResource(AnywhereProfile, anywhereProfile)
+ c.newResource(IRARole, iraRole)
+
+ c.rs.defineOutputFromAtt(outputs.RemoteNodesTrustAnchorARN, TrustAnchor, "TrustAnchorArn", true, func(v string) error {
+ return nil
+ })
+ c.rs.defineOutputFromAtt(outputs.RemoteNodesAnywhereProfileARN, AnywhereProfile, "ProfileArn", true, func(v string) error {
+ return nil
+ })
+ c.rs.defineOutputFromAtt(outputs.RemoteNodesRoleARN, IRARole, "Arn", true, func(v string) error {
+ c.spec.RemoteNetworkConfig.IAM.RoleARN = &v
+ return nil
+ })
+}
+
+func (c *ClusterResourceSet) addSSM() {
+ role := &gfniam.Role{
+ AssumeRolePolicyDocument: cft.MakeAssumeRolePolicyDocumentForServices(
+ MakeServiceRef("SSM"),
+ ),
+ Policies: []gfniam.Role_Policy{
+ ssmRolePolicy,
+ eksDescribeClusterPolicy,
+ },
+ ManagedPolicyArns: gfnt.NewSlice(makePolicyARNs([]string{
+ iamPolicyAmazonSSMManagedInstanceCore,
+ iamPolicyAmazonEC2ContainerRegistryReadOnly,
+ }...)...),
+ }
+ c.newResource(SSMRole, role)
+ c.rs.defineOutputFromAtt(outputs.RemoteNodesRoleARN, SSMRole, "Arn", true, func(v string) error {
+ c.spec.RemoteNetworkConfig.IAM.RoleARN = &v
+ return nil
+ })
+}
+
+func (c *ClusterResourceSet) addResourcesForRemoteNodesRole() {
+ c.rs.withIAM = true
+ switch *c.spec.RemoteNetworkConfig.IAM.Provider {
+ case api.SSMProvider:
+ c.addSSM()
+ case api.IRAProvider:
+ c.addIAMRolesAnywhere()
+ default:
+ // Validations should ensure this is never reached
+ }
+}
+
+func (c *ClusterResourceSet) addResourcesForIAM() {
+ c.rs.withIAM = false
+ c.rs.withNamedIAM = false
+
+ if !api.IsSetAndNonEmptyString(c.spec.IAM.ServiceRoleARN) {
+ c.addResourcesForServiceRole()
+ } else {
+ c.rs.defineOutputWithoutCollector(outputs.ClusterServiceRoleARN, c.spec.IAM.ServiceRoleARN, true)
+ }
+
+ if c.spec.HasRemoteNetworkingConfigured() {
+ if !api.IsSetAndNonEmptyString(c.spec.RemoteNetworkConfig.IAM.RoleARN) {
+ c.addResourcesForRemoteNodesRole()
+ } else {
+ c.rs.defineOutputWithoutCollector(outputs.RemoteNodesRoleARN, c.spec.RemoteNetworkConfig.IAM.RoleARN, true)
+ }
+ }
+}
+
// WithIAM states, if IAM roles will be created or not
func (n *NodeGroupResourceSet) WithIAM() bool {
return n.rs.withIAM
diff --git a/pkg/cfn/builder/karpenter_test.go b/pkg/cfn/builder/karpenter_test.go
index 39605cd7ce..00d876b794 100644
--- a/pkg/cfn/builder/karpenter_test.go
+++ b/pkg/cfn/builder/karpenter_test.go
@@ -75,7 +75,9 @@ var expectedTemplate = `{
"aws": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
},
"aws-cn": {
"EC2": "ec2.amazonaws.com.cn",
@@ -95,7 +97,9 @@ var expectedTemplate = `{
"aws-us-gov": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
}
}
},
@@ -217,7 +221,9 @@ var expectedTemplateWithPermissionBoundary = `{
"aws": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
},
"aws-cn": {
"EC2": "ec2.amazonaws.com.cn",
@@ -237,7 +243,9 @@ var expectedTemplateWithPermissionBoundary = `{
"aws-us-gov": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
}
}
},
@@ -360,7 +368,9 @@ var expectedTemplateWithSpotInterruptionQueue = `{
"aws": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
},
"aws-cn": {
"EC2": "ec2.amazonaws.com.cn",
@@ -380,7 +390,9 @@ var expectedTemplateWithSpotInterruptionQueue = `{
"aws-us-gov": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
}
}
},
diff --git a/pkg/cfn/builder/testdata/nodegroup_access_entry/1.json b/pkg/cfn/builder/testdata/nodegroup_access_entry/1.json
index 19b7dbf438..d6b612e00f 100644
--- a/pkg/cfn/builder/testdata/nodegroup_access_entry/1.json
+++ b/pkg/cfn/builder/testdata/nodegroup_access_entry/1.json
@@ -6,7 +6,9 @@
"aws": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
},
"aws-cn": {
"EC2": "ec2.amazonaws.com.cn",
@@ -26,7 +28,9 @@
"aws-us-gov": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
}
}
},
diff --git a/pkg/cfn/builder/testdata/nodegroup_access_entry/2.json b/pkg/cfn/builder/testdata/nodegroup_access_entry/2.json
index 610b6aa45d..01bf3728f0 100644
--- a/pkg/cfn/builder/testdata/nodegroup_access_entry/2.json
+++ b/pkg/cfn/builder/testdata/nodegroup_access_entry/2.json
@@ -6,7 +6,9 @@
"aws": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
},
"aws-cn": {
"EC2": "ec2.amazonaws.com.cn",
@@ -26,7 +28,9 @@
"aws-us-gov": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
}
}
},
diff --git a/pkg/cfn/builder/testdata/nodegroup_access_entry/3.json b/pkg/cfn/builder/testdata/nodegroup_access_entry/3.json
index bcfd6635dd..79ff343817 100644
--- a/pkg/cfn/builder/testdata/nodegroup_access_entry/3.json
+++ b/pkg/cfn/builder/testdata/nodegroup_access_entry/3.json
@@ -6,7 +6,9 @@
"aws": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
},
"aws-cn": {
"EC2": "ec2.amazonaws.com.cn",
@@ -26,7 +28,9 @@
"aws-us-gov": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
}
}
},
diff --git a/pkg/cfn/builder/vpc_ipv4.go b/pkg/cfn/builder/vpc_ipv4.go
index 300812d3de..99fbb14e38 100644
--- a/pkg/cfn/builder/vpc_ipv4.go
+++ b/pkg/cfn/builder/vpc_ipv4.go
@@ -3,6 +3,7 @@ package builder
import (
"context"
"fmt"
+ "strconv"
"strings"
"github.com/weaveworks/eksctl/pkg/awsapi"
@@ -30,6 +31,7 @@ type IPv4VPCResourceSet struct {
ec2API awsapi.EC2
vpcID *gfnt.Value
subnetDetails *SubnetDetails
+ azToRTMap map[string]*gfnt.Value
extendForOutposts bool
}
@@ -60,6 +62,7 @@ func NewIPv4VPCResourceSet(rs *resourceSet, clusterConfig *api.ClusterConfig, ec
controlPlaneOnOutposts: clusterConfig.IsControlPlaneOnOutposts(),
autoMode: clusterConfig.IsAutoModeEnabled(),
},
+ azToRTMap: make(map[string]*gfnt.Value),
extendForOutposts: extendForOutposts,
}
}
@@ -82,20 +85,36 @@ func (v *IPv4VPCResourceSet) addResources() error {
EnableDnsHostnames: gfnt.True(),
})
+ // Add base private networking config, which is common to all eksctl created VPCs i.e.
+ // - Private Subnets
+ // - Private Route Tables
+ v.addPrivateRouteTables()
+ v.subnetDetails.Private = v.addSubnets(nil, api.SubnetTopologyPrivate, vpc.Subnets.Private)
+
if v.clusterConfig.IsFullyPrivate() {
- v.noNAT()
- v.subnetDetails.Private = v.addSubnets(nil, api.SubnetTopologyPrivate, vpc.Subnets.Private)
+ // if the cluster if fully private, we have already added all required resources
return nil
}
- refIG := v.rs.newResource("InternetGateway", &gfnec2.InternetGateway{})
- vpcGA := "VPCGatewayAttachment"
+ // Add full private networking config i.e.
+ // - NAT Gateway(s)
+ // - Hybrid Nodes Networking
+ if err := v.addNATGateways(); err != nil {
+ return err
+ }
+ if v.clusterConfig.HasRemoteNetworkingConfigured() {
+ v.addHybridNodesNetworking()
+ }
- v.rs.newResource(vpcGA, &gfnec2.VPCGatewayAttachment{
+ // Add public networking config i.e.
+ // - Public Subnets
+ // - Public Route Table
+ // - Internet Gateway
+ refIG := v.rs.newResource("InternetGateway", &gfnec2.InternetGateway{})
+ v.rs.newResource("VPCGatewayAttachment", &gfnec2.VPCGatewayAttachment{
InternetGatewayId: refIG,
VpcId: v.vpcID,
})
-
refPublicRT := v.rs.newResource("PublicRouteTable", &gfnec2.RouteTable{
VpcId: v.vpcID,
})
@@ -109,24 +128,17 @@ func (v *IPv4VPCResourceSet) addResources() error {
RouteTableId: refPublicRT,
DestinationIpv6CidrBlock: gfnt.NewString(InternetIPv6CIDR),
GatewayId: refIG,
- AWSCloudFormationDependsOn: []string{vpcGA},
+ AWSCloudFormationDependsOn: []string{"VPCGatewayAttachment"},
})
}
-
v.rs.newResource("PublicSubnetRoute", &gfnec2.Route{
RouteTableId: refPublicRT,
DestinationCidrBlock: gfnt.NewString(InternetCIDR),
GatewayId: refIG,
- AWSCloudFormationDependsOn: []string{vpcGA},
+ AWSCloudFormationDependsOn: []string{"VPCGatewayAttachment"},
})
-
v.subnetDetails.Public = v.addSubnets(refPublicRT, api.SubnetTopologyPublic, vpc.Subnets.Public)
- if err := v.addNATGateways(); err != nil {
- return err
- }
-
- v.subnetDetails.Private = v.addSubnets(nil, api.SubnetTopologyPrivate, vpc.Subnets.Private)
if vpc.LocalZoneSubnets != nil {
if len(vpc.LocalZoneSubnets.Public) > 0 {
v.subnetDetails.PublicLocalZone = v.addSubnets(refPublicRT, api.SubnetTopologyPublic, vpc.LocalZoneSubnets.Public)
@@ -319,6 +331,86 @@ func (v *IPv4VPCResourceSet) addSubnets(refRT *gfnt.Value, topology api.SubnetTo
return subnetResources
}
+func (v *IPv4VPCResourceSet) getPrivateRouteTableRefForAZ(az string) *gfnt.Value {
+ return v.azToRTMap[az]
+}
+
+func (v *IPv4VPCResourceSet) addPrivateRouteTables() {
+ forEachPrivateSubnet(v.clusterConfig.VPC, func(subnetAlias string) {
+ subnetAZResourceName := makeAZResourceName(subnetAlias)
+
+ refRT := v.rs.newResource("PrivateRouteTable"+subnetAZResourceName, &gfnec2.RouteTable{
+ VpcId: v.vpcID,
+ })
+ v.azToRTMap[subnetAZResourceName] = refRT
+
+ v.rs.newResource("RouteTableAssociationPrivate"+subnetAZResourceName, &gfnec2.SubnetRouteTableAssociation{
+ SubnetId: gfnt.MakeRef("SubnetPrivate" + subnetAZResourceName),
+ RouteTableId: refRT,
+ })
+ })
+}
+
+func (v *IPv4VPCResourceSet) addHybridNodesNetworking() {
+ switch {
+ case v.clusterConfig.RemoteNetworkConfig.VPCGatewayID.IsVirtualPrivateGateway():
+ VGWRef := gfnt.NewString(string(*v.clusterConfig.RemoteNetworkConfig.VPCGatewayID))
+ v.rs.newResource("VirtualPrivateGatewayAttachment", &gfnec2.VPCGatewayAttachment{
+ VpnGatewayId: VGWRef,
+ VpcId: v.vpcID,
+ })
+ forEachPrivateSubnet(v.clusterConfig.VPC, func(subnetAlias string) {
+ subnetAZResourceName := makeAZResourceName(subnetAlias)
+ for i, remoteNetworkCIDR := range v.clusterConfig.RemoteNetworkConfig.ToRemoteNetworksPool() {
+ v.rs.newResource("VGWPrivateSubnetRoute"+strconv.Itoa(i)+subnetAZResourceName, &gfnec2.Route{
+ RouteTableId: v.getPrivateRouteTableRefForAZ(subnetAZResourceName),
+ DestinationCidrBlock: gfnt.NewString(remoteNetworkCIDR),
+ GatewayId: VGWRef,
+ AWSCloudFormationDependsOn: []string{"VirtualPrivateGatewayAttachment"},
+ })
+ }
+ })
+
+ case v.clusterConfig.RemoteNetworkConfig.VPCGatewayID.IsTransitGateway():
+ privateSubnetRefs := []*gfnt.Value{}
+ forEachPrivateSubnet(v.clusterConfig.VPC, func(subnetAlias string) {
+ subnetAZResourceName := makeAZResourceName(subnetAlias)
+ privateSubnetRefs = append(privateSubnetRefs, gfnt.MakeRef("SubnetPrivate"+subnetAZResourceName))
+ })
+ TGWRef := gfnt.NewString(string(*v.clusterConfig.RemoteNetworkConfig.VPCGatewayID))
+ v.rs.newResource("TransitGatewayAttachment", &gfnec2.TransitGatewayAttachment{
+ //nolint: golint
+ TransitGatewayId: TGWRef,
+ VpcId: v.vpcID,
+ SubnetIds: gfnt.NewSlice(privateSubnetRefs...),
+ })
+ forEachPrivateSubnet(v.clusterConfig.VPC, func(subnetAlias string) {
+ subnetAZResourceName := makeAZResourceName(subnetAlias)
+ for i, remoteNetworkCIDR := range v.clusterConfig.RemoteNetworkConfig.ToRemoteNetworksPool() {
+ v.rs.newResource("TGWPrivateSubnetRoute"+strconv.Itoa(i)+subnetAZResourceName, &gfnec2.Route{
+ RouteTableId: v.getPrivateRouteTableRefForAZ(subnetAZResourceName),
+ DestinationCidrBlock: gfnt.NewString(remoteNetworkCIDR),
+ TransitGatewayId: TGWRef,
+ AWSCloudFormationDependsOn: []string{"TransitGatewayAttachment"},
+ })
+ }
+ })
+ default:
+ return
+ }
+}
+
+func forEachPrivateSubnet(clusterVPC *api.ClusterVPC, fn func(subnetAlias string)) {
+ for subnetAlias := range clusterVPC.Subnets.Private {
+ fn(subnetAlias)
+ }
+ if clusterVPC.LocalZoneSubnets != nil {
+ for subnetAlias := range clusterVPC.LocalZoneSubnets.Private {
+ fn(subnetAlias)
+ }
+ }
+}
+
func (v *IPv4VPCResourceSet) addNATGateways() error {
switch *v.clusterConfig.VPC.NAT.Gateway {
case api.ClusterHighlyAvailableNAT:
@@ -326,7 +418,7 @@ func (v *IPv4VPCResourceSet) addNATGateways() error {
case api.ClusterSingleNAT:
v.singleNAT()
case api.ClusterDisableNAT:
- v.noNAT()
+ // Nothing to do
default:
// TODO validate this before starting to add resources
return fmt.Errorf("%s is not a valid NAT gateway mode", *v.clusterConfig.VPC.NAT.Gateway)
@@ -395,50 +487,17 @@ func (v *IPv4VPCResourceSet) singleNAT() {
SubnetId: gfnt.MakeRef("SubnetPublic" + firstUpperAZ),
})
- forEachNATSubnet(v.clusterConfig.VPC, func(subnetAlias string) {
+ forEachPrivateSubnet(v.clusterConfig.VPC, func(subnetAlias string) {
subnetAZResourceName := makeAZResourceName(subnetAlias)
- refRT := v.rs.newResource("PrivateRouteTable"+subnetAZResourceName, &gfnec2.RouteTable{
- VpcId: v.vpcID,
- })
-
v.rs.newResource("NATPrivateSubnetRoute"+subnetAZResourceName, &gfnec2.Route{
- RouteTableId: refRT,
+ RouteTableId: v.getPrivateRouteTableRefForAZ(subnetAZResourceName),
DestinationCidrBlock: gfnt.NewString(InternetCIDR),
NatGatewayId: refNG,
})
- v.rs.newResource("RouteTableAssociationPrivate"+subnetAZResourceName, &gfnec2.SubnetRouteTableAssociation{
- SubnetId: gfnt.MakeRef("SubnetPrivate" + subnetAZResourceName),
- RouteTableId: refRT,
- })
})
}
-func (v *IPv4VPCResourceSet) noNAT() {
- forEachNATSubnet(v.clusterConfig.VPC, func(subnetAlias string) {
- subnetAZResourceName := makeAZResourceName(subnetAlias)
-
- refRT := v.rs.newResource("PrivateRouteTable"+subnetAZResourceName, &gfnec2.RouteTable{
- VpcId: v.vpcID,
- })
- v.rs.newResource("RouteTableAssociationPrivate"+subnetAZResourceName, &gfnec2.SubnetRouteTableAssociation{
- SubnetId: gfnt.MakeRef("SubnetPrivate" + subnetAZResourceName),
- RouteTableId: refRT,
- })
- })
-}
-
-func forEachNATSubnet(clusterVPC *api.ClusterVPC, fn func(subnetAlias string)) {
- for subnetAlias := range clusterVPC.Subnets.Private {
- fn(subnetAlias)
- }
- if clusterVPC.LocalZoneSubnets != nil {
- for subnetAlias := range clusterVPC.LocalZoneSubnets.Private {
- fn(subnetAlias)
- }
- }
-}
-
func makeAZResourceName(subnetAZ string) string {
return strings.ToUpper(strings.ReplaceAll(subnetAZ, "-", ""))
}
diff --git a/pkg/cfn/manager/fakes/fake_nodegroup_resource_set.go b/pkg/cfn/manager/fakes/fake_nodegroup_resource_set.go
new file mode 100644
index 0000000000..5a10b8f7e9
--- /dev/null
+++ b/pkg/cfn/manager/fakes/fake_nodegroup_resource_set.go
@@ -0,0 +1,387 @@
+// Code generated by counterfeiter. DO NOT EDIT.
+package fakes
+
+import (
+ "context"
+ "sync"
+
+ "github.com/aws/aws-sdk-go-v2/service/cloudformation/types"
+ "github.com/weaveworks/eksctl/pkg/cfn/manager"
+)
+
+type FakeNodeGroupResourceSet struct {
+ AddAllResourcesStub func(context.Context) error
+ addAllResourcesMutex sync.RWMutex
+ addAllResourcesArgsForCall []struct {
+ arg1 context.Context
+ }
+ addAllResourcesReturns struct {
+ result1 error
+ }
+ addAllResourcesReturnsOnCall map[int]struct {
+ result1 error
+ }
+ GetAllOutputsStub func(types.Stack) error
+ getAllOutputsMutex sync.RWMutex
+ getAllOutputsArgsForCall []struct {
+ arg1 types.Stack
+ }
+ getAllOutputsReturns struct {
+ result1 error
+ }
+ getAllOutputsReturnsOnCall map[int]struct {
+ result1 error
+ }
+ RenderJSONStub func() ([]byte, error)
+ renderJSONMutex sync.RWMutex
+ renderJSONArgsForCall []struct {
+ }
+ renderJSONReturns struct {
+ result1 []byte
+ result2 error
+ }
+ renderJSONReturnsOnCall map[int]struct {
+ result1 []byte
+ result2 error
+ }
+ WithIAMStub func() bool
+ withIAMMutex sync.RWMutex
+ withIAMArgsForCall []struct {
+ }
+ withIAMReturns struct {
+ result1 bool
+ }
+ withIAMReturnsOnCall map[int]struct {
+ result1 bool
+ }
+ WithNamedIAMStub func() bool
+ withNamedIAMMutex sync.RWMutex
+ withNamedIAMArgsForCall []struct {
+ }
+ withNamedIAMReturns struct {
+ result1 bool
+ }
+ withNamedIAMReturnsOnCall map[int]struct {
+ result1 bool
+ }
+ invocations map[string][][]interface{}
+ invocationsMutex sync.RWMutex
+}
+
+func (fake *FakeNodeGroupResourceSet) AddAllResources(arg1 context.Context) error {
+ fake.addAllResourcesMutex.Lock()
+ ret, specificReturn := fake.addAllResourcesReturnsOnCall[len(fake.addAllResourcesArgsForCall)]
+ fake.addAllResourcesArgsForCall = append(fake.addAllResourcesArgsForCall, struct {
+ arg1 context.Context
+ }{arg1})
+ stub := fake.AddAllResourcesStub
+ fakeReturns := fake.addAllResourcesReturns
+ fake.recordInvocation("AddAllResources", []interface{}{arg1})
+ fake.addAllResourcesMutex.Unlock()
+ if stub != nil {
+ return stub(arg1)
+ }
+ if specificReturn {
+ return ret.result1
+ }
+ return fakeReturns.result1
+}
+
+func (fake *FakeNodeGroupResourceSet) AddAllResourcesCallCount() int {
+ fake.addAllResourcesMutex.RLock()
+ defer fake.addAllResourcesMutex.RUnlock()
+ return len(fake.addAllResourcesArgsForCall)
+}
+
+func (fake *FakeNodeGroupResourceSet) AddAllResourcesCalls(stub func(context.Context) error) {
+ fake.addAllResourcesMutex.Lock()
+ defer fake.addAllResourcesMutex.Unlock()
+ fake.AddAllResourcesStub = stub
+}
+
+func (fake *FakeNodeGroupResourceSet) AddAllResourcesArgsForCall(i int) context.Context {
+ fake.addAllResourcesMutex.RLock()
+ defer fake.addAllResourcesMutex.RUnlock()
+ argsForCall := fake.addAllResourcesArgsForCall[i]
+ return argsForCall.arg1
+}
+
+func (fake *FakeNodeGroupResourceSet) AddAllResourcesReturns(result1 error) {
+ fake.addAllResourcesMutex.Lock()
+ defer fake.addAllResourcesMutex.Unlock()
+ fake.AddAllResourcesStub = nil
+ fake.addAllResourcesReturns = struct {
+ result1 error
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) AddAllResourcesReturnsOnCall(i int, result1 error) {
+ fake.addAllResourcesMutex.Lock()
+ defer fake.addAllResourcesMutex.Unlock()
+ fake.AddAllResourcesStub = nil
+ if fake.addAllResourcesReturnsOnCall == nil {
+ fake.addAllResourcesReturnsOnCall = make(map[int]struct {
+ result1 error
+ })
+ }
+ fake.addAllResourcesReturnsOnCall[i] = struct {
+ result1 error
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) GetAllOutputs(arg1 types.Stack) error {
+ fake.getAllOutputsMutex.Lock()
+ ret, specificReturn := fake.getAllOutputsReturnsOnCall[len(fake.getAllOutputsArgsForCall)]
+ fake.getAllOutputsArgsForCall = append(fake.getAllOutputsArgsForCall, struct {
+ arg1 types.Stack
+ }{arg1})
+ stub := fake.GetAllOutputsStub
+ fakeReturns := fake.getAllOutputsReturns
+ fake.recordInvocation("GetAllOutputs", []interface{}{arg1})
+ fake.getAllOutputsMutex.Unlock()
+ if stub != nil {
+ return stub(arg1)
+ }
+ if specificReturn {
+ return ret.result1
+ }
+ return fakeReturns.result1
+}
+
+func (fake *FakeNodeGroupResourceSet) GetAllOutputsCallCount() int {
+ fake.getAllOutputsMutex.RLock()
+ defer fake.getAllOutputsMutex.RUnlock()
+ return len(fake.getAllOutputsArgsForCall)
+}
+
+func (fake *FakeNodeGroupResourceSet) GetAllOutputsCalls(stub func(types.Stack) error) {
+ fake.getAllOutputsMutex.Lock()
+ defer fake.getAllOutputsMutex.Unlock()
+ fake.GetAllOutputsStub = stub
+}
+
+func (fake *FakeNodeGroupResourceSet) GetAllOutputsArgsForCall(i int) types.Stack {
+ fake.getAllOutputsMutex.RLock()
+ defer fake.getAllOutputsMutex.RUnlock()
+ argsForCall := fake.getAllOutputsArgsForCall[i]
+ return argsForCall.arg1
+}
+
+func (fake *FakeNodeGroupResourceSet) GetAllOutputsReturns(result1 error) {
+ fake.getAllOutputsMutex.Lock()
+ defer fake.getAllOutputsMutex.Unlock()
+ fake.GetAllOutputsStub = nil
+ fake.getAllOutputsReturns = struct {
+ result1 error
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) GetAllOutputsReturnsOnCall(i int, result1 error) {
+ fake.getAllOutputsMutex.Lock()
+ defer fake.getAllOutputsMutex.Unlock()
+ fake.GetAllOutputsStub = nil
+ if fake.getAllOutputsReturnsOnCall == nil {
+ fake.getAllOutputsReturnsOnCall = make(map[int]struct {
+ result1 error
+ })
+ }
+ fake.getAllOutputsReturnsOnCall[i] = struct {
+ result1 error
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) RenderJSON() ([]byte, error) {
+ fake.renderJSONMutex.Lock()
+ ret, specificReturn := fake.renderJSONReturnsOnCall[len(fake.renderJSONArgsForCall)]
+ fake.renderJSONArgsForCall = append(fake.renderJSONArgsForCall, struct {
+ }{})
+ stub := fake.RenderJSONStub
+ fakeReturns := fake.renderJSONReturns
+ fake.recordInvocation("RenderJSON", []interface{}{})
+ fake.renderJSONMutex.Unlock()
+ if stub != nil {
+ return stub()
+ }
+ if specificReturn {
+ return ret.result1, ret.result2
+ }
+ return fakeReturns.result1, fakeReturns.result2
+}
+
+func (fake *FakeNodeGroupResourceSet) RenderJSONCallCount() int {
+ fake.renderJSONMutex.RLock()
+ defer fake.renderJSONMutex.RUnlock()
+ return len(fake.renderJSONArgsForCall)
+}
+
+func (fake *FakeNodeGroupResourceSet) RenderJSONCalls(stub func() ([]byte, error)) {
+ fake.renderJSONMutex.Lock()
+ defer fake.renderJSONMutex.Unlock()
+ fake.RenderJSONStub = stub
+}
+
+func (fake *FakeNodeGroupResourceSet) RenderJSONReturns(result1 []byte, result2 error) {
+ fake.renderJSONMutex.Lock()
+ defer fake.renderJSONMutex.Unlock()
+ fake.RenderJSONStub = nil
+ fake.renderJSONReturns = struct {
+ result1 []byte
+ result2 error
+ }{result1, result2}
+}
+
+func (fake *FakeNodeGroupResourceSet) RenderJSONReturnsOnCall(i int, result1 []byte, result2 error) {
+ fake.renderJSONMutex.Lock()
+ defer fake.renderJSONMutex.Unlock()
+ fake.RenderJSONStub = nil
+ if fake.renderJSONReturnsOnCall == nil {
+ fake.renderJSONReturnsOnCall = make(map[int]struct {
+ result1 []byte
+ result2 error
+ })
+ }
+ fake.renderJSONReturnsOnCall[i] = struct {
+ result1 []byte
+ result2 error
+ }{result1, result2}
+}
+
+func (fake *FakeNodeGroupResourceSet) WithIAM() bool {
+ fake.withIAMMutex.Lock()
+ ret, specificReturn := fake.withIAMReturnsOnCall[len(fake.withIAMArgsForCall)]
+ fake.withIAMArgsForCall = append(fake.withIAMArgsForCall, struct {
+ }{})
+ stub := fake.WithIAMStub
+ fakeReturns := fake.withIAMReturns
+ fake.recordInvocation("WithIAM", []interface{}{})
+ fake.withIAMMutex.Unlock()
+ if stub != nil {
+ return stub()
+ }
+ if specificReturn {
+ return ret.result1
+ }
+ return fakeReturns.result1
+}
+
+func (fake *FakeNodeGroupResourceSet) WithIAMCallCount() int {
+ fake.withIAMMutex.RLock()
+ defer fake.withIAMMutex.RUnlock()
+ return len(fake.withIAMArgsForCall)
+}
+
+func (fake *FakeNodeGroupResourceSet) WithIAMCalls(stub func() bool) {
+ fake.withIAMMutex.Lock()
+ defer fake.withIAMMutex.Unlock()
+ fake.WithIAMStub = stub
+}
+
+func (fake *FakeNodeGroupResourceSet) WithIAMReturns(result1 bool) {
+ fake.withIAMMutex.Lock()
+ defer fake.withIAMMutex.Unlock()
+ fake.WithIAMStub = nil
+ fake.withIAMReturns = struct {
+ result1 bool
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) WithIAMReturnsOnCall(i int, result1 bool) {
+ fake.withIAMMutex.Lock()
+ defer fake.withIAMMutex.Unlock()
+ fake.WithIAMStub = nil
+ if fake.withIAMReturnsOnCall == nil {
+ fake.withIAMReturnsOnCall = make(map[int]struct {
+ result1 bool
+ })
+ }
+ fake.withIAMReturnsOnCall[i] = struct {
+ result1 bool
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) WithNamedIAM() bool {
+ fake.withNamedIAMMutex.Lock()
+ ret, specificReturn := fake.withNamedIAMReturnsOnCall[len(fake.withNamedIAMArgsForCall)]
+ fake.withNamedIAMArgsForCall = append(fake.withNamedIAMArgsForCall, struct {
+ }{})
+ stub := fake.WithNamedIAMStub
+ fakeReturns := fake.withNamedIAMReturns
+ fake.recordInvocation("WithNamedIAM", []interface{}{})
+ fake.withNamedIAMMutex.Unlock()
+ if stub != nil {
+ return stub()
+ }
+ if specificReturn {
+ return ret.result1
+ }
+ return fakeReturns.result1
+}
+
+func (fake *FakeNodeGroupResourceSet) WithNamedIAMCallCount() int {
+ fake.withNamedIAMMutex.RLock()
+ defer fake.withNamedIAMMutex.RUnlock()
+ return len(fake.withNamedIAMArgsForCall)
+}
+
+func (fake *FakeNodeGroupResourceSet) WithNamedIAMCalls(stub func() bool) {
+ fake.withNamedIAMMutex.Lock()
+ defer fake.withNamedIAMMutex.Unlock()
+ fake.WithNamedIAMStub = stub
+}
+
+func (fake *FakeNodeGroupResourceSet) WithNamedIAMReturns(result1 bool) {
+ fake.withNamedIAMMutex.Lock()
+ defer fake.withNamedIAMMutex.Unlock()
+ fake.WithNamedIAMStub = nil
+ fake.withNamedIAMReturns = struct {
+ result1 bool
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) WithNamedIAMReturnsOnCall(i int, result1 bool) {
+ fake.withNamedIAMMutex.Lock()
+ defer fake.withNamedIAMMutex.Unlock()
+ fake.WithNamedIAMStub = nil
+ if fake.withNamedIAMReturnsOnCall == nil {
+ fake.withNamedIAMReturnsOnCall = make(map[int]struct {
+ result1 bool
+ })
+ }
+ fake.withNamedIAMReturnsOnCall[i] = struct {
+ result1 bool
+ }{result1}
+}
+
+func (fake *FakeNodeGroupResourceSet) Invocations() map[string][][]interface{} {
+ fake.invocationsMutex.RLock()
+ defer fake.invocationsMutex.RUnlock()
+ fake.addAllResourcesMutex.RLock()
+ defer fake.addAllResourcesMutex.RUnlock()
+ fake.getAllOutputsMutex.RLock()
+ defer fake.getAllOutputsMutex.RUnlock()
+ fake.renderJSONMutex.RLock()
+ defer fake.renderJSONMutex.RUnlock()
+ fake.withIAMMutex.RLock()
+ defer fake.withIAMMutex.RUnlock()
+ fake.withNamedIAMMutex.RLock()
+ defer fake.withNamedIAMMutex.RUnlock()
+ copiedInvocations := map[string][][]interface{}{}
+ for key, value := range fake.invocations {
+ copiedInvocations[key] = value
+ }
+ return copiedInvocations
+}
+
+func (fake *FakeNodeGroupResourceSet) recordInvocation(key string, args []interface{}) {
+ fake.invocationsMutex.Lock()
+ defer fake.invocationsMutex.Unlock()
+ if fake.invocations == nil {
+ fake.invocations = map[string][][]interface{}{}
+ }
+ if fake.invocations[key] == nil {
+ fake.invocations[key] = [][]interface{}{}
+ }
+ fake.invocations[key] = append(fake.invocations[key], args)
+}
+
+var _ manager.NodeGroupResourceSet = new(FakeNodeGroupResourceSet)
diff --git a/pkg/cfn/outputs/api.go b/pkg/cfn/outputs/api.go
index d667427e81..6b65f0b950 100644
--- a/pkg/cfn/outputs/api.go
+++ b/pkg/cfn/outputs/api.go
@@ -34,6 +34,12 @@ const (
ClusterServiceRoleARN = "ServiceRoleARN"
ClusterFeatureNATMode = "FeatureNATMode"
+ // outputs for remote nodes
+ RemoteNodesRoleARN = "RemoteNodesRoleARN"
+ RemoteNodesIntermediateRoleARN = "RemoteNodesIntermediateRoleARN"
+ RemoteNodesTrustAnchorARN = "RemoteNodesTrustAnchorARN"
+ RemoteNodesAnywhereProfileARN = "RemoteNodesAnywhereProfileARN"
+
// outputs from nodegroup stack
NodeGroupInstanceRoleARN = "InstanceRoleARN"
NodeGroupInstanceProfileARN = "InstanceProfileARN"
diff --git a/pkg/cfn/template/iam_helpers.go b/pkg/cfn/template/iam_helpers.go
index 4977b86dbf..6ca76e0f4c 100644
--- a/pkg/cfn/template/iam_helpers.go
+++ b/pkg/cfn/template/iam_helpers.go
@@ -79,3 +79,15 @@ func MakeAssumeRolePolicyDocumentForPodIdentity() MapOfInterfaces {
},
})
}
+
+// MakeAssumeRolePolicyDocumentForServices constructs a trust policy for given services with given conditions and extra actions
+func MakeAssumeRolePolicyDocumentForServicesWithConditionsAndActions(condition MapOfInterfaces, extraActions []string, services ...*gfn.Value) MapOfInterfaces {
+ return MakePolicyDocument(MapOfInterfaces{
+ "Effect": "Allow",
+ "Action": append([]string{"sts:AssumeRole"}, extraActions...),
+ "Condition": condition,
+ "Principal": map[string][]*gfn.Value{
+ "Service": services,
+ },
+ })
+}
diff --git a/pkg/cfn/template/testdata/nodegroup-example-1.json b/pkg/cfn/template/testdata/nodegroup-example-1.json
index e7dc35897e..4c2548db1d 100644
--- a/pkg/cfn/template/testdata/nodegroup-example-1.json
+++ b/pkg/cfn/template/testdata/nodegroup-example-1.json
@@ -6,7 +6,9 @@
"aws": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
},
"aws-cn": {
"EC2": "ec2.amazonaws.com.cn",
@@ -16,7 +18,9 @@
"aws-us-gov": {
"EC2": "ec2.amazonaws.com",
"EKS": "eks.amazonaws.com",
- "EKSFargatePods": "eks-fargate-pods.amazonaws.com"
+ "EKSFargatePods": "eks-fargate-pods.amazonaws.com",
+ "IRA": "rolesanywhere.amazonaws.com",
+ "SSM": "ssm.amazonaws.com"
}
}
},
diff --git a/userdocs/mkdocs.yml b/userdocs/mkdocs.yml
index 7a025268f3..44534c5ef8 100644
--- a/userdocs/mkdocs.yml
+++ b/userdocs/mkdocs.yml
@@ -175,6 +175,7 @@ nav:
- usage/container-runtime.md
- usage/windows-worker-nodes.md
- usage/nodegroup-additional-volume-mappings.md
+ - usage/hybrid-nodes.md
- usage/eksctl-karpenter.md
- usage/eksctl-anywhere.md
- GitOps:
diff --git a/userdocs/src/usage/hybrid-nodes.md b/userdocs/src/usage/hybrid-nodes.md
new file mode 100644
index 0000000000..d5cb69afac
--- /dev/null
+++ b/userdocs/src/usage/hybrid-nodes.md
@@ -0,0 +1,97 @@
+# EKS Hybrid Nodes
+
+AWS EKS introduces Hybrid Nodes, a new feature that enables you to run on-premises and edge applications on customer-managed infrastructure with the same AWS EKS clusters, features, and tools you use in the AWS Cloud. AWS EKS Hybird Nodes brings an AWS-managed Kubernetes experience to on-premises environments for customers to simplify and standardize how you run applications across on-premises, edge and cloud environments. Read more at [EKS Hybrid Nodes][eks-hybrid-nodes].
+
+To facilitate support for this feature, eksctl introduces a new top-level field called `remoteNetworkConfig`. Any Hybrid Nodes related configuration shall be set up via this field, as part of the config file; there are no CLI flags counterparts. Additionally, at launch, any remote network config can only be set up during cluster creation and cannot be updated afterwards. This means, you won't be able to update existing clusters to use Hybrid Nodes.
+
+The `remoteNetworkConfig` section of the config file allows you to setup the two core areas when it comes to joining remote nodes to you EKS clusters: **networking** and **credentials**.
+
+## Networking
+
+EKS Hybrid Nodes is flexible to your preferred method of connecting your on-premises network(s) to an AWS VPC. There are several [documented options](https://docs.aws.amazon.com/whitepapers/latest/aws-vpc-connectivity-options/network-to-amazon-vpc-connectivity-options.html) available, including AWS Site-to-Site VPN and AWS Direct Connect, and you can choose the method that best fits your use case. In most of the methods you might choose, your VPC will be attached to either a virtual private gateway (VGW) or a transit gateway (TGW). If you rely on eksctl to create a VPC for you, eksctl will also configure, **within the scope of your VPC**, any networking related pre-requisites in order to facilitate communication between your EKS control plane and the remote nodes i.e.
+
+- ingress/egress SG rules
+- routes in the private subnets' route tables
+- the VPC gateway attachment to the given TGW or VGW
+
+Example config file:
+
+```yaml
+remoteNetworkConfig:
+ vpcGatewayID: tgw-xxxx # either VGW or TGW to be attached to your VPC
+ remoteNodeNetworks:
+ # eksctl will create, behind the scenes, SG rules, routes, and a VPC gateway attachment,
+ # to facilitate communication between remote network(s) and EKS control plane, via the attached gateway
+ - cidrs: ["10.80.146.0/24"]
+ remotePodNetworks:
+ - cidrs: ["10.86.30.0/23"]
+```
+
+If your connectivity method of choice does not involve using a TGW or VGW, you must not rely on eksctl to create the VPC for you, and instead provide a pre-existing one. On a related note, if you are using a pre-existing VPC, eksctl won't make any amendments to it, and ensuring all networking requirements are in place falls under your responsibility.
+
+???+ note
+ eksctl does not setup any networking infrastructure outside your AWS VPC (i.e. any infrastructure from VGW/TGW to the remote networks)
+
+## Credentials
+
+EKS Hybrid Nodes use the AWS IAM Authenticator and temporary IAM credentials provisioned by either **AWS SSM** or **AWS IAM Roles Anywhere**
+to authenticate with the EKS cluster. Similar to the self-managed nodegroups, if not otherwise provided, eksctl will create for you a Hybrid Nodes IAM Role to be assumed by the remote nodes. Additioanlly, when using IAM Roles Anywhere as your credentials provider, eksctl will setup a profile, and trust anchor based on a given certificate authority bundle (`iam.caBundleCert`) e.g.
+
+```yaml
+remoteNetworkConfig:
+ iam:
+ # the provider for temporary IAM credentials. Default is SSM.
+ provider: IRA
+ # the certificate authority bundle that serves as the root of trust,
+ # used to validate the X.509 certificates provided by your nodes.
+ # can only be set when provider is IAMRolesAnywhere.
+ caBundleCert: xxxx
+ ```
+
+The ARN of the Hybrid Nodes Role created by eksctl is needed later in the process of joining your remote nodes to the cluster, to setup `NodeConfig` for `nodeadm`, and to create activations (if using SSM). To fetch it, use:
+
+```bash
+aws cloudformation describe-stacks \
+ --stack-name eksctl-eksctl is now fully maintained by AWS. For more details check out
eksctl Support Status Update.
-
- eksctl now supports Auto Mode.
-
+ eksctl now supports Auto Mode.
+
+ eksctl now supports Hybrid Nodes.
+
eksctl now supports Cluster creation flexibility for networking add-ons.