Skip to content

Commit f65632a

Browse files
[system test] [doc] MultipleClusterOperatorsST, NamespaceDeletionReco… (#12006)
Signed-off-by: see-quick <[email protected]> Signed-off-by: Maros Orsak <[email protected]> Co-authored-by: PaulRMellor <[email protected]>
1 parent 7f22096 commit f65632a

9 files changed

+265
-56
lines changed
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# MultipleClusterOperatorsST
2+
3+
**Description:** Test suite for verifying multiple Cluster Operator deployment scenarios, including resource selectors, leader election, and metrics collection across different namespace configurations.
4+
5+
**Before test execution steps:**
6+
7+
| Step | Action | Result |
8+
| - | - | - |
9+
| 1. | Skip this test suite if using Helm or OLM installation. | The test suite only runs with YAML-based installations. |
10+
11+
**Labels:**
12+
13+
* [kafka](labels/kafka.md)
14+
15+
<hr style="border:1px solid">
16+
17+
## testMultipleCOsInDifferentNamespaces
18+
19+
**Description:** This test verifies how two Cluster Operators operate resources deployed in namespaces watched by both operators, and how operands can be transitioned from one Cluster Operator to another using label selectors.
20+
21+
**Steps:**
22+
23+
| Step | Action | Result |
24+
| - | - | - |
25+
| 1. | Deploy two Cluster Operators in separate namespaces, both watching all namespaces. | Both Cluster Operators are successfully deployed. |
26+
| 2. | Set up scrapers and metric collectors for both Cluster Operators. | Scraper Pods and metrics collectors are configured and ready. |
27+
| 3. | Create namespace for test resources. | Namespace `multiple-co-cluster-test` is created and set as default. |
28+
| 4. | Deploy Kafka without operator selector label. | The `Kafka` resource is created but no pods are deployed because it is not managed by any operator. |
29+
| 5. | Verify Kafka metrics are absent in both operators. | Metric `strimzi_resource` for Kafka is null or zero in both Cluster Operators. |
30+
| 6. | Add label selector pointing to first Cluster Operator. | Kafka is deployed and managed by the first Cluster Operator. |
31+
| 7. | Deploy `KafkaConnect` and `KafkaConnector` with a label selecting the first Cluster Operator. | Both resources are successfully deployed and managed by the first Cluster Operator. |
32+
| 8. | Produce and consume messages using Sink Connector. | Messages are produced to topic and consumed by the Connector successfully. |
33+
| 9. | Switch Kafka management to the second Cluster Operator by changing the label. | Kafka management transfers to second operator, as confirmed by updated metrics. |
34+
| 10. | Verify metrics for all operands on both operators. | Metric `strimzi_resource` shows correct counts for each Cluster Operator. |
35+
36+
**Labels:**
37+
38+
* [kafka](labels/kafka.md)
39+
* [connect](labels/connect.md)
40+
* [metrics](labels/metrics.md)
41+
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# NamespaceDeletionRecoveryST
2+
3+
**Description:** Test suite for verifying Kafka cluster recovery after namespace deletion. Tests cover scenarios with and without `KafkaTopic` resources available, using persistent volumes with the `Retain` reclaim policy. Note: This suite requires `StorageClass` with a local provisioner on Minikube.
4+
5+
**Before test execution steps:**
6+
7+
| Step | Action | Result |
8+
| - | - | - |
9+
| 1. | Create a `StorageClass` with the `Retain` reclaim policy. | `StorageClass` is created with `WaitForFirstConsumer` volume binding mode. |
10+
11+
**After test execution steps:**
12+
13+
| Step | Action | Result |
14+
| - | - | - |
15+
| 1. | Clean up orphaned persistent volumes. | All Kafka-related persistent volumes are deleted. |
16+
17+
**Labels:**
18+
19+
* [kafka](labels/kafka.md)
20+
21+
<hr style="border:1px solid">
22+
23+
## testTopicAvailable
24+
25+
**Description:** This test verifies Kafka cluster recovery when all `KafkaTopic` resources are available after namespace deletion, including internal topics. The test recreates `KafkaTopic` resources first, then deploys the Kafka cluster and verifies that existing data is preserved.
26+
27+
**Steps:**
28+
29+
| Step | Action | Result |
30+
| - | - | - |
31+
| 1. | Prepare the test environment with a Kafka cluster and `KafkaTopic`. | The Kafka cluster is deployed with persistent storage, and the topic contains test data. |
32+
| 2. | Store the list of all `KafkaTopic` and `PersistentVolumeClaim` resources. | All `KafkaTopic` and `PersistentVolumeClaim` resources are captured for recovery. |
33+
| 3. | Delete and recreate the namespace. | Namespace is deleted and recreated successfully. |
34+
| 4. | Recreate `PersistentVolumeClaims` and rebind `PersistentVolumes` resources. | The `PersistentVolumeClaim` resources are recreated and bound to the existing `PersistentVolumeClaim` resources. |
35+
| 5. | Recreate the Cluster Operator in the namespace. | The Cluster Operator is deployed and ready. |
36+
| 6. | Recreate all `KafkaTopic` resources. | All `KafkaTopic` resources are recreated successfully. |
37+
| 7. | Deploy the Kafka cluster with persistent storage. | Kafka cluster is deployed and becomes ready. |
38+
| 8. | Verify data recovery by producing and consuming messages. | Messages can be consumed, confirming data persisted through the recovery process. |
39+
40+
**Labels:**
41+
42+
* [kafka](labels/kafka.md)
43+
44+
45+
## testTopicNotAvailable
46+
47+
**Description:** This test verifies Kafka cluster recovery when `KafkaTopic` resources are not available after namespace deletion. The test deploys Kafka without the Topic Operator first to preserve existing topics, then enables Topic Operator after cluster becomes stable.
48+
49+
**Steps:**
50+
51+
| Step | Action | Result |
52+
| - | - | - |
53+
| 1. | Prepare test environment with a Kafka cluster and test data. | The Kafka cluster is deployed with persistent storage, and the topic contains test data. |
54+
| 2. | Store the cluster ID and list of `PersistentVolumeClaim` resources. | The Cluster ID and `PersistentVolumeClaim` list are captured for recovery. |
55+
| 3. | List the current topics in Kafka cluster. | The topic list is logged for verification. |
56+
| 4. | Delete and recreate the namespace. | The namespace is deleted and recreated successfully. |
57+
| 5. | Recreate `PersistentVolumeClaims` and rebind `PersistentVolumes` resources. | The `PersistentVolumeClaim` resources are recreated and bound to the existing `PersistentVolume` resources. |
58+
| 6. | Recreate the Cluster Operator in the namespace. | The Cluster Operator is deployed and ready. |
59+
| 7. | Deploy Kafka without the Topic Operator using the pause annotation. | The Kafka cluster is created without Topic Operator to prevent topic deletion. |
60+
| 8. | Patch the Kafka status with original cluster ID. | The Cluster ID is restored to match the original cluster. |
61+
| 9. | Unpause Kafka reconciliation. | The Kafka cluster becomes ready and operational. |
62+
| 10. | Enable the Topic Operator by updating the `Kafka` resource. | The Topic Operator is deployed and starts managing topics. |
63+
| 11. | Verify data recovery by producing and consuming messages. | Messages can be consumed, confirming that topics and data were preserved. |
64+
65+
**Labels:**
66+
67+
* [kafka](labels/kafka.md)
68+
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# NamespaceRbacScopeOperatorST
2+
3+
**Description:** Test suite for verifying the namespace-scoped RBAC deployment mode for the Cluster Operator, ensuring that `Role` resources are created instead of `ClusterRole` resources when operating in namespace-scoped mode.
4+
5+
**Before test execution steps:**
6+
7+
| Step | Action | Result |
8+
| - | - | - |
9+
| 1. | Skip this test suite if using OLM or Helm installation. | This test suite only runs with YAML-based installations. |
10+
11+
**Labels:**
12+
13+
* [kafka](labels/kafka.md)
14+
15+
<hr style="border:1px solid">
16+
17+
## testNamespacedRbacScopeDeploysRoles
18+
19+
**Description:** This test verifies that when the Cluster Operator is deployed with namespace-scoped RBAC, it creates `Role` resources instead of `ClusterRole` resources, ensuring proper isolation and avoiding cluster-wide permissions.
20+
21+
**Steps:**
22+
23+
| Step | Action | Result |
24+
| - | - | - |
25+
| 1. | Deploy the Cluster Operator with namespace-scoped RBAC configuration. | The Cluster Operator is deployed with RBAC type set to `NAMESPACE`. |
26+
| 2. | Deploy a Kafka cluster with broker and controller node pools. | The Kafka cluster is deployed and becomes ready. |
27+
| 3. | Verify no `ClusterRole` resources with Strimzi labels exist. | No `ClusterRole` resources labeled with `app=strimzi` are found in the cluster. |
28+
29+
**Labels:**
30+
31+
* [kafka](labels/kafka.md)
32+

development-docs/systemtests/labels/connect.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ maintaining data consistency and availability in a streaming ecosystem.
3434
- [testKafkaConnectorWithoutClusterConfig](../io.strimzi.systemtest.operators.CustomResourceStatusST.md)
3535
- [testMountingSecretAndConfigMapAsVolumesAndEnvVars](../io.strimzi.systemtest.connect.ConnectST.md)
3636
- [testMultiNodeKafkaConnectWithConnectorCreation](../io.strimzi.systemtest.connect.ConnectST.md)
37+
- [testMultipleCOsInDifferentNamespaces](../io.strimzi.systemtest.operators.MultipleClusterOperatorsST.md)
3738
- [testPushIntoImageStream](../io.strimzi.systemtest.connect.ConnectBuilderST.md)
3839
- [testScaleConnectAndConnectorSubresource](../io.strimzi.systemtest.connect.ConnectST.md)
3940
- [testScaleConnectWithConnectorToZero](../io.strimzi.systemtest.connect.ConnectST.md)

development-docs/systemtests/labels/kafka.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,10 +75,12 @@ These tests are crucial to ensure that Kafka clusters can handle production work
7575
- [testMessagesTlsScramShaWithPredefinedPassword](../io.strimzi.systemtest.kafka.listeners.ListenersST.md)
7676
- [testMirrorMaker2Metrics](../io.strimzi.systemtest.metrics.MetricsST.md)
7777
- [testMixtureOfExternalListeners](../io.strimzi.systemtest.kafka.listeners.MultipleListenersST.md)
78+
- [testMultipleCOsInDifferentNamespaces](../io.strimzi.systemtest.operators.MultipleClusterOperatorsST.md)
7879
- [testMultipleInternal](../io.strimzi.systemtest.kafka.listeners.MultipleListenersST.md)
7980
- [testMultipleLoadBalancers](../io.strimzi.systemtest.kafka.listeners.MultipleListenersST.md)
8081
- [testMultipleNodePorts](../io.strimzi.systemtest.kafka.listeners.MultipleListenersST.md)
8182
- [testMultipleRoutes](../io.strimzi.systemtest.kafka.listeners.MultipleListenersST.md)
83+
- [testNamespacedRbacScopeDeploysRoles](../io.strimzi.systemtest.operators.NamespaceRbacScopeOperatorST.md)
8284
- [testNodePoolsAdditionAndRemoval](../io.strimzi.systemtest.kafka.KafkaNodePoolST.md)
8385
- [testNodePoolsRolesChanging](../io.strimzi.systemtest.kafka.KafkaNodePoolST.md)
8486
- [testNodePort](../io.strimzi.systemtest.kafka.listeners.ListenersST.md)
@@ -100,6 +102,8 @@ These tests are crucial to ensure that Kafka clusters can handle production work
100102
- [testSimpleDynamicConfiguration](../io.strimzi.systemtest.kafka.dynamicconfiguration.DynamicConfST.md)
101103
- [testTieredStorageWithAivenFileSystemPlugin](../io.strimzi.systemtest.kafka.TieredStorageST.md)
102104
- [testTieredStorageWithAivenS3Plugin](../io.strimzi.systemtest.kafka.TieredStorageST.md)
105+
- [testTopicAvailable](../io.strimzi.systemtest.operators.NamespaceDeletionRecoveryST.md)
106+
- [testTopicNotAvailable](../io.strimzi.systemtest.operators.NamespaceDeletionRecoveryST.md)
103107
- [testUpdateToExternalListenerCausesRollingRestart](../io.strimzi.systemtest.kafka.dynamicconfiguration.DynamicConfST.md)
104108
- [testUpdateToExternalListenerCausesRollingRestartUsingExternalClients](../io.strimzi.systemtest.kafka.dynamicconfiguration.DynamicConfST.md)
105109
- [testUserOperatorMetrics](../io.strimzi.systemtest.metrics.MetricsST.md)

development-docs/systemtests/labels/metrics.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,5 @@ Proper verification of metrics is crucial for observability, debugging, and perf
1919
- [testKafkaMetrics](../io.strimzi.systemtest.metrics.MetricsST.md)
2020
- [testKafkaMetricsSettings](../io.strimzi.systemtest.metrics.MetricsST.md)
2121
- [testMirrorMaker2Metrics](../io.strimzi.systemtest.metrics.MetricsST.md)
22+
- [testMultipleCOsInDifferentNamespaces](../io.strimzi.systemtest.operators.MultipleClusterOperatorsST.md)
2223
- [testUserOperatorMetrics](../io.strimzi.systemtest.metrics.MetricsST.md)

systemtest/src/test/java/io/strimzi/systemtest/operators/MultipleClusterOperatorsST.java

Lines changed: 35 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,11 @@
66

77
import io.fabric8.kubernetes.api.model.EnvVar;
88
import io.fabric8.kubernetes.api.model.rbac.ClusterRoleBinding;
9+
import io.skodjob.annotations.Desc;
10+
import io.skodjob.annotations.Label;
11+
import io.skodjob.annotations.Step;
12+
import io.skodjob.annotations.SuiteDoc;
13+
import io.skodjob.annotations.TestDoc;
914
import io.skodjob.testframe.resources.KubeResourceManager;
1015
import io.strimzi.api.kafka.model.connect.KafkaConnect;
1116
import io.strimzi.api.kafka.model.connector.KafkaConnector;
@@ -19,6 +24,7 @@
1924
import io.strimzi.systemtest.Environment;
2025
import io.strimzi.systemtest.TestConstants;
2126
import io.strimzi.systemtest.annotations.IsolatedTest;
27+
import io.strimzi.systemtest.docs.TestDocsLabels;
2228
import io.strimzi.systemtest.kafkaclients.internalClients.KafkaClients;
2329
import io.strimzi.systemtest.performance.gather.collectors.BaseMetricsCollector;
2430
import io.strimzi.systemtest.resources.CrdClients;
@@ -65,6 +71,15 @@
6571
import static org.junit.jupiter.api.Assumptions.assumeTrue;
6672

6773
@Tag(REGRESSION)
74+
@SuiteDoc(
75+
description = @Desc("Test suite for verifying multiple Cluster Operator deployment scenarios, including resource selectors, leader election, and metrics collection across different namespace configurations."),
76+
beforeTestSteps = {
77+
@Step(value = "Skip this test suite if using Helm or OLM installation.", expected = "The test suite only runs with YAML-based installations.")
78+
},
79+
labels = {
80+
@Label(value = TestDocsLabels.KAFKA)
81+
}
82+
)
6883
public class MultipleClusterOperatorsST extends AbstractST {
6984

7085
private static final Logger LOGGER = LogManager.getLogger(MultipleClusterOperatorsST.class);
@@ -85,43 +100,29 @@ public class MultipleClusterOperatorsST extends AbstractST {
85100
public static final Map<String, String> FIRST_CO_SELECTOR = Collections.singletonMap("app.kubernetes.io/operator", FIRST_CO_NAME);
86101
public static final Map<String, String> SECOND_CO_SELECTOR = Collections.singletonMap("app.kubernetes.io/operator", SECOND_CO_NAME);
87102

88-
/**
89-
* @description This test case checks how two Cluster Operators operates operands deployed in namespace watched by both of these operators, and transition
90-
* of operand from one Cluster Operator to another.
91-
*
92-
* @steps
93-
* 1. - Deploy two Cluster Operators - first to the first namespace and second to the second namespace - both watching all namespaces.
94-
* - Cluster Operators are successfully deployed.
95-
* 2. - Set up scrapers and metric collectors for both of Cluster Operators.
96-
* - Mentioned resources are successfully set up.
97-
* 3. - Create and Set namespace 'multiple-co-cluster-test' as default, because all resources from now on will be deployed here.
98-
* - Namespace is created and set as default.
99-
* 4. - Deploy Kafka operand in default namespace without label 'app.kubernetes.io/operator'.
100-
* - Kafka is not deployed.
101-
* 5. - Verify state of metric 'strimzi_resource' for Kafka, which holds number of operated Kafka operands.
102-
* - Metric 'strimzi_resource' exposed by Cluster Operators is not present indicating no Kafka resource is being operated.
103-
* 6. - Modify Kafka CustomResource, by changing its label 'app.kubernetes.io/operator' to point to the first Cluster Operator
104-
* - Kafka is deployed and operated by the first Cluster Operator.
105-
* 7. - Deploy Kafka Connect (with label 'app.kubernetes.io/operator' pointing the first Cluster Operator) and Kafka Connector
106-
* - Both operands are successfully deployed, and managed by the first Cluster Operator.
107-
* 8. - Produce messages into the Kafka Topic and consume with Sink Connector.
108-
* - Messages are produced and later handled correctly by the Connector.
109-
* 9. - Management of Kafka is switched to the second Cluster Operator by modifying value of label 'app.kubernetes.io/operator'.
110-
* - Management is modified, later confirmed by the expected change in the value of metric 'strimzi_resource' for kind Kafka on both Cluster Operators.
111-
* 10. - Verify 'strimzi_resource' of each Cluster Operator for all deployed operands.
112-
* - Metric 'strimzi_resource' indicates expected number of operated resources for both Cluster Operators.
113-
*
114-
* @usecase
115-
* - cluster-operator-metrics
116-
* - cluster-operator-watcher
117-
* - connect
118-
* - kafka
119-
* - labels
120-
* - metrics
121-
*/
122103
@IsolatedTest
123104
@Tag(CONNECT)
124105
@Tag(CONNECT_COMPONENTS)
106+
@TestDoc(
107+
description = @Desc("This test verifies how two Cluster Operators operate resources deployed in namespaces watched by both operators, and how operands can be transitioned from one Cluster Operator to another using label selectors."),
108+
steps = {
109+
@Step(value = "Deploy two Cluster Operators in separate namespaces, both watching all namespaces.", expected = "Both Cluster Operators are successfully deployed."),
110+
@Step(value = "Set up scrapers and metric collectors for both Cluster Operators.", expected = "Scraper Pods and metrics collectors are configured and ready."),
111+
@Step(value = "Create namespace for test resources.", expected = "Namespace `multiple-co-cluster-test` is created and set as default."),
112+
@Step(value = "Deploy Kafka without operator selector label.", expected = "The `Kafka` resource is created but no pods are deployed because it is not managed by any operator."),
113+
@Step(value = "Verify Kafka metrics are absent in both operators.", expected = "Metric `strimzi_resource` for Kafka is null or zero in both Cluster Operators."),
114+
@Step(value = "Add label selector pointing to first Cluster Operator.", expected = "Kafka is deployed and managed by the first Cluster Operator."),
115+
@Step(value = "Deploy `KafkaConnect` and `KafkaConnector` with a label selecting the first Cluster Operator.", expected = "Both resources are successfully deployed and managed by the first Cluster Operator."),
116+
@Step(value = "Produce and consume messages using Sink Connector.", expected = "Messages are produced to topic and consumed by the Connector successfully."),
117+
@Step(value = "Switch Kafka management to the second Cluster Operator by changing the label.", expected = "Kafka management transfers to second operator, as confirmed by updated metrics."),
118+
@Step(value = "Verify metrics for all operands on both operators.", expected = "Metric `strimzi_resource` shows correct counts for each Cluster Operator.")
119+
},
120+
labels = {
121+
@Label(value = TestDocsLabels.KAFKA),
122+
@Label(value = TestDocsLabels.CONNECT),
123+
@Label(value = TestDocsLabels.METRICS)
124+
}
125+
)
125126
void testMultipleCOsInDifferentNamespaces() {
126127
// Strimzi is deployed with cluster-wide access in this class STRIMZI_RBAC_SCOPE=NAMESPACE won't work
127128
assumeFalse(Environment.isNamespaceRbacScope());

0 commit comments

Comments
 (0)