Skip to content

Commit 9978183

Browse files
Kirill-GarbarKirill-Garbar
authored
Add auth enabler (#204)
fixes #160 ## Changelog ### Added * [chart+kustomize] * what: Added get secrets permissions for operator namespace. * why: Necessary for operator to get secrets with certificates. * [api] * what: Added helper functions to let getting settings easier. * why: Unify getting settings throughout the project code. * [api] * what: Added ServerTrustedCASecret to spec. * why: Necessary to mount this certificate to let operator trust etcd cluster. * **[controller]** * **what: Added functionality to disable and enable auth, add root role, root user.** * **why: Necessary for the customer.** ### Changed * [api] * what: Adjusted field descriptions for security fields. * why: Necessary to let customers know where we expect created secrets with certificates. * [etcdcluster_controller_test] * what: Commented autotests that reconcile twice and set sts ready. * why: Not clear how to handle failed tests. It is not supposed to set ready status when it is not ready. Creating mocks for every function that uses etcdClient will take much time in the future
1 parent c0cb38b commit 9978183

File tree

20 files changed

+889
-125
lines changed

20 files changed

+889
-125
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ vendor
2222

2323
# editor and IDE paraphernalia
2424
.idea
25+
.vscode
2526
*.swp
2627
*.swo
2728
*~

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ mod-tidy: ## Run go mod tidy against code.
7474

7575
.PHONY: test
7676
test: manifests generate fmt vet envtest ## Run tests.
77-
echo "Check for kubernetes version $(K8S_VERSION_TRIMMED_V) in $(ENVTEST)"
77+
@echo "Check for kubernetes version $(K8S_VERSION_TRIMMED_V) in $(ENVTEST)"
7878
@$(ENVTEST) list | grep -q $(K8S_VERSION_TRIMMED_V)
7979
KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(K8S_VERSION_TRIMMED_V) --bin-dir $(LOCALBIN) -p path)" go test $$(go list ./... | grep -v /e2e) -coverprofile cover.out
8080

‎api/v1alpha1/etcdcluster_types.go‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,18 @@ func (r *EtcdCluster) CalculateQuorumSize() int {
9898
return int(*r.Spec.Replicas)/2 + 1
9999
}
100100

101+
func (c *EtcdCluster) IsClientSecurityEnabled() bool {
102+
return c.Spec.Security != nil && c.Spec.Security.TLS.ClientSecret != ""
103+
}
104+
105+
func (c *EtcdCluster) IsServerSecurityEnabled() bool {
106+
return c.Spec.Security != nil && c.Spec.Security.TLS.ServerSecret != ""
107+
}
108+
109+
func (c *EtcdCluster) IsServerTrustedCADefined() bool {
110+
return c.Spec.Security != nil && c.Spec.Security.TLS.ServerTrustedCASecret != ""
111+
}
112+
101113
// +kubebuilder:object:root=true
102114

103115
// EtcdClusterList contains a list of EtcdCluster
@@ -174,24 +186,36 @@ type SecuritySpec struct {
174186
// Section for user-managed tls certificates
175187
// +optional
176188
TLS TLSSpec `json:"tls,omitempty"`
189+
// Section to enable etcd auth
190+
EnableAuth bool `json:"enableAuth,omitempty"`
177191
}
178192

179193
// TLSSpec defines user-managed certificates names.
180194
type TLSSpec struct {
181-
// Trusted CA certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt field in the secret.
195+
// Trusted CA certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have ca.crt field in the secret.
196+
// This secret must be created in the namespace with etcdCluster CR.
182197
// +optional
183198
PeerTrustedCASecret string `json:"peerTrustedCASecret,omitempty"`
184199
// Certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt and tls.key fields in the secret.
200+
// This secret must be created in the namespace with etcdCluster CR.
185201
// +optional
186202
PeerSecret string `json:"peerSecret,omitempty"`
203+
// Trusted CA for etcd server certificates for client-server communication. Is necessary to set trust between operator and etcd.
204+
// It is expected to have ca.crt field in the secret. If it is not specified, then insecure communication will be used.
205+
// This secret must be created in the namespace with etcdCluster CR.
206+
// +optional
207+
ServerTrustedCASecret string `json:"serverTrustedCASecret,omitempty"`
187208
// Server certificate secret to secure client-server communication. Is provided to the client who connects to etcd by client port (2379 by default).
188209
// It is expected to have tls.crt and tls.key fields in the secret.
210+
// This secret must be created in the namespace with etcdCluster CR.
189211
// +optional
190212
ServerSecret string `json:"serverSecret,omitempty"`
191-
// Trusted CA for client certificates that are provided by client to etcd. It is expected to have tls.crt field in the secret.
213+
// Trusted CA for client certificates that are provided by client to etcd. It is expected to have ca.crt field in the secret.
214+
// This secret must be created in the namespace with etcdCluster CR.
192215
// +optional
193216
ClientTrustedCASecret string `json:"clientTrustedCASecret,omitempty"`
194217
// Client certificate for etcd-operator to do maintenance. It is expected to have tls.crt and tls.key fields in the secret.
218+
// This secret must be created in the namespace with etcdCluster CR.
195219
// +optional
196220
ClientSecret string `json:"clientSecret,omitempty"`
197221
}

‎api/v1alpha1/etcdcluster_types_test.go‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,105 @@ var _ = Context("CalculateQuorumSize", func() {
2020
Expect(etcdCluster.CalculateQuorumSize()).To(Equal(3))
2121
})
2222
})
23+
24+
var _ = Describe("Aux Functions", func() {
25+
26+
Context("When running IsClientSecurityEnabled function", func() {
27+
It("should return true if ClientSecret is set", func() {
28+
cluster := EtcdCluster{
29+
Spec: EtcdClusterSpec{
30+
Security: &SecuritySpec{
31+
TLS: TLSSpec{
32+
ClientSecret: "some-client-secret",
33+
},
34+
},
35+
},
36+
}
37+
Expect(cluster.IsClientSecurityEnabled()).To(BeTrue())
38+
})
39+
40+
It("should return false if ClientSecret is not set", func() {
41+
cluster := EtcdCluster{
42+
Spec: EtcdClusterSpec{
43+
Security: &SecuritySpec{
44+
TLS: TLSSpec{},
45+
},
46+
},
47+
}
48+
Expect(cluster.IsClientSecurityEnabled()).To(BeFalse())
49+
})
50+
51+
It("should return false if Security is nil", func() {
52+
cluster := &EtcdCluster{
53+
Spec: EtcdClusterSpec{},
54+
}
55+
Expect(cluster.IsClientSecurityEnabled()).To(BeFalse())
56+
})
57+
})
58+
59+
Context("When running IsServerSecurityEnabled function", func() {
60+
It("should return true if ServerSecret is set", func() {
61+
cluster := &EtcdCluster{
62+
Spec: EtcdClusterSpec{
63+
Security: &SecuritySpec{
64+
TLS: TLSSpec{
65+
ServerSecret: "some-server-secret",
66+
},
67+
},
68+
},
69+
}
70+
Expect(cluster.IsServerSecurityEnabled()).To(BeTrue())
71+
})
72+
73+
It("should return false if ServerSecret is not set", func() {
74+
cluster := &EtcdCluster{
75+
Spec: EtcdClusterSpec{
76+
Security: &SecuritySpec{
77+
TLS: TLSSpec{},
78+
},
79+
},
80+
}
81+
Expect(cluster.IsServerSecurityEnabled()).To(BeFalse())
82+
})
83+
84+
It("should return false if Security is nil", func() {
85+
cluster := &EtcdCluster{
86+
Spec: EtcdClusterSpec{},
87+
}
88+
Expect(cluster.IsServerSecurityEnabled()).To(BeFalse())
89+
})
90+
})
91+
92+
Context("When running IsServerTrustedCADefined function", func() {
93+
It("should return true if ServerTrustedCASecret is set", func() {
94+
cluster := &EtcdCluster{
95+
Spec: EtcdClusterSpec{
96+
Security: &SecuritySpec{
97+
TLS: TLSSpec{
98+
ServerTrustedCASecret: "some-ca-secret",
99+
},
100+
},
101+
},
102+
}
103+
Expect(cluster.IsServerTrustedCADefined()).To(BeTrue())
104+
})
105+
106+
It("should return false if ServerTrustedCASecret is not set", func() {
107+
cluster := &EtcdCluster{
108+
Spec: EtcdClusterSpec{
109+
Security: &SecuritySpec{
110+
TLS: TLSSpec{},
111+
},
112+
},
113+
}
114+
Expect(cluster.IsServerTrustedCADefined()).To(BeFalse())
115+
})
116+
117+
It("should return false if Security is nil", func() {
118+
cluster := &EtcdCluster{
119+
Spec: EtcdClusterSpec{},
120+
}
121+
Expect(cluster.IsServerTrustedCADefined()).To(BeFalse())
122+
})
123+
})
124+
})

‎api/v1alpha1/etcdcluster_webhook.go‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -281,6 +281,15 @@ func (r *EtcdCluster) validateSecurity() field.ErrorList {
281281
)
282282
}
283283

284+
if security.EnableAuth && (security.TLS.ClientSecret == "" || security.TLS.ServerSecret == "") {
285+
286+
allErrors = append(allErrors, field.Invalid(
287+
field.NewPath("spec", "security"),
288+
security.TLS,
289+
"if auth is enabled, client secret and server secret must be provided"),
290+
)
291+
}
292+
284293
if len(allErrors) > 0 {
285294
return allErrors
286295
}

‎api/v1alpha1/etcdcluster_webhook_test.go‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,66 @@ var _ = Describe("EtcdCluster Webhook", func() {
213213
}
214214
}
215215
})
216+
217+
It("Shouldn't reject if auth is enabled and security client certs are defined", func() {
218+
localCluster := etcdCluster.DeepCopy()
219+
localCluster.Spec.Security = &SecuritySpec{
220+
EnableAuth: true,
221+
TLS: TLSSpec{
222+
ClientTrustedCASecret: "test-client-trusted-ca-cert",
223+
ClientSecret: "test-client-cert",
224+
ServerSecret: "test-server-cert",
225+
},
226+
}
227+
err := localCluster.validateSecurity()
228+
Expect(err).To(BeNil())
229+
})
230+
231+
It("Should reject if auth is enabled and one of client and server certs is defined", func() {
232+
localCluster := etcdCluster.DeepCopy()
233+
localCluster.Spec.Security = &SecuritySpec{
234+
EnableAuth: true,
235+
TLS: TLSSpec{
236+
ClientSecret: "test-client-cert",
237+
ClientTrustedCASecret: "test-client-trusted-ca-cert",
238+
},
239+
}
240+
err := localCluster.validateSecurity()
241+
if Expect(err).NotTo(BeNil()) {
242+
expectedFieldErr := field.Invalid(
243+
field.NewPath("spec", "security"),
244+
localCluster.Spec.Security.TLS,
245+
"if auth is enabled, client secret and server secret must be provided",
246+
)
247+
if Expect(err).To(HaveLen(1)) {
248+
Expect(*(err[0])).To(Equal(*expectedFieldErr))
249+
}
250+
}
251+
252+
})
253+
254+
It("Should reject if auth is enabled and one of client and server certs is defined", func() {
255+
localCluster := etcdCluster.DeepCopy()
256+
localCluster.Spec.Security = &SecuritySpec{
257+
EnableAuth: true,
258+
TLS: TLSSpec{
259+
ServerSecret: "test-server-cert",
260+
},
261+
}
262+
err := localCluster.validateSecurity()
263+
if Expect(err).NotTo(BeNil()) {
264+
expectedFieldErr := field.Invalid(
265+
field.NewPath("spec", "security"),
266+
localCluster.Spec.Security.TLS,
267+
"if auth is enabled, client secret and server secret must be provided",
268+
)
269+
if Expect(err).To(HaveLen(1)) {
270+
Expect(*(err[0])).To(Equal(*expectedFieldErr))
271+
}
272+
}
273+
274+
})
275+
216276
})
217277

218278
Context("Validate PDB", func() {

‎charts/etcd-operator/crds/etcd-cluster.yaml‎

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -202,25 +202,43 @@ spec:
202202
security:
203203
description: Security describes security settings of etcd (authentication, certificates, rbac)
204204
properties:
205+
enableAuth:
206+
description: Section to enable etcd auth
207+
type: boolean
205208
tls:
206209
description: Section for user-managed tls certificates
207210
properties:
208211
clientSecret:
209-
description: Client certificate for etcd-operator to do maintenance. It is expected to have tls.crt and tls.key fields in the secret.
212+
description: |-
213+
Client certificate for etcd-operator to do maintenance. It is expected to have tls.crt and tls.key fields in the secret.
214+
This secret must be created in the namespace with etcdCluster CR.
210215
type: string
211216
clientTrustedCASecret:
212-
description: Trusted CA for client certificates that are provided by client to etcd. It is expected to have tls.crt field in the secret.
217+
description: |-
218+
Trusted CA for client certificates that are provided by client to etcd. It is expected to have ca.crt field in the secret.
219+
This secret must be created in the namespace with etcdCluster CR.
213220
type: string
214221
peerSecret:
215-
description: Certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt and tls.key fields in the secret.
222+
description: |-
223+
Certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt and tls.key fields in the secret.
224+
This secret must be created in the namespace with etcdCluster CR.
216225
type: string
217226
peerTrustedCASecret:
218-
description: Trusted CA certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt field in the secret.
227+
description: |-
228+
Trusted CA certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have ca.crt field in the secret.
229+
This secret must be created in the namespace with etcdCluster CR.
219230
type: string
220231
serverSecret:
221232
description: |-
222233
Server certificate secret to secure client-server communication. Is provided to the client who connects to etcd by client port (2379 by default).
223234
It is expected to have tls.crt and tls.key fields in the secret.
235+
This secret must be created in the namespace with etcdCluster CR.
236+
type: string
237+
serverTrustedCASecret:
238+
description: |-
239+
Trusted CA for etcd server certificates for client-server communication. Is necessary to set trust between operator and etcd.
240+
It is expected to have ca.crt field in the secret. If it is not specified, then insecure communication will be used.
241+
This secret must be created in the namespace with etcdCluster CR.
224242
type: string
225243
type: object
226244
type: object

‎charts/etcd-operator/templates/workload/deployment.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,12 @@ spec:
5858
- configMapRef:
5959
name: {{ include "etcd-operator.fullname" . }}-env
6060
{{- end }}
61+
env:
62+
- name: POD_NAMESPACE
63+
valueFrom:
64+
fieldRef:
65+
apiVersion: v1
66+
fieldPath: metadata.namespace
6167
volumeMounts:
6268
- mountPath: /tmp/k8s-webhook-server/serving-certs
6369
name: cert

‎config/crd/bases/etcd.aenix.io_etcdclusters.yaml‎

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -192,25 +192,43 @@ spec:
192192
security:
193193
description: Security describes security settings of etcd (authentication, certificates, rbac)
194194
properties:
195+
enableAuth:
196+
description: Section to enable etcd auth
197+
type: boolean
195198
tls:
196199
description: Section for user-managed tls certificates
197200
properties:
198201
clientSecret:
199-
description: Client certificate for etcd-operator to do maintenance. It is expected to have tls.crt and tls.key fields in the secret.
202+
description: |-
203+
Client certificate for etcd-operator to do maintenance. It is expected to have tls.crt and tls.key fields in the secret.
204+
This secret must be created in the namespace with etcdCluster CR.
200205
type: string
201206
clientTrustedCASecret:
202-
description: Trusted CA for client certificates that are provided by client to etcd. It is expected to have tls.crt field in the secret.
207+
description: |-
208+
Trusted CA for client certificates that are provided by client to etcd. It is expected to have ca.crt field in the secret.
209+
This secret must be created in the namespace with etcdCluster CR.
203210
type: string
204211
peerSecret:
205-
description: Certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt and tls.key fields in the secret.
212+
description: |-
213+
Certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt and tls.key fields in the secret.
214+
This secret must be created in the namespace with etcdCluster CR.
206215
type: string
207216
peerTrustedCASecret:
208-
description: Trusted CA certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have tls.crt field in the secret.
217+
description: |-
218+
Trusted CA certificate secret to secure peer-to-peer communication between etcd nodes. It is expected to have ca.crt field in the secret.
219+
This secret must be created in the namespace with etcdCluster CR.
209220
type: string
210221
serverSecret:
211222
description: |-
212223
Server certificate secret to secure client-server communication. Is provided to the client who connects to etcd by client port (2379 by default).
213224
It is expected to have tls.crt and tls.key fields in the secret.
225+
This secret must be created in the namespace with etcdCluster CR.
226+
type: string
227+
serverTrustedCASecret:
228+
description: |-
229+
Trusted CA for etcd server certificates for client-server communication. Is necessary to set trust between operator and etcd.
230+
It is expected to have ca.crt field in the secret. If it is not specified, then insecure communication will be used.
231+
This secret must be created in the namespace with etcdCluster CR.
214232
type: string
215233
type: object
216234
type: object

‎config/default/manager_auth_proxy_patch.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,3 +37,4 @@ spec:
3737
- "--health-probe-bind-address=:8081"
3838
- "--metrics-bind-address=127.0.0.1:8080"
3939
- "--leader-elect"
40+
- "--log-level=debug"

0 commit comments

Comments
 (0)