Controller panics on GKE cluster deletion #1454
Description
/kind bug
What steps did you take and what happened:
During GKE cluster deletion, the controller panics on a nil pointer dereference.
I0320 15:26:05.343670 1 gcpmanagedcluster_controller.go:229] "Reconciling Delete GCPManagedCluster" controller="gcpmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedCluster" GCPManagedCluster="default-l7fz79/cluster-gke" namespace="default-l7fz79" name="cluster-gke" reconcileID="fe225b66-4459-4afe-b832-1ad02846cc39" controller="gcpmanagedcluster" action="delete"
I0320 15:26:05.343766 1 gcpmanagedcluster_controller.go:232] "GCPManagedControlPlane not deleted yet, retry later" controller="gcpmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedCluster" GCPManagedCluster="default-l7fz79/cluster-gke" namespace="default-l7fz79" name="cluster-gke" reconcileID="fe225b66-4459-4afe-b832-1ad02846cc39" controller="gcpmanagedcluster" action="delete"
I0320 15:26:05.390085 1 gcpmanagedcluster_controller.go:268] "GCPManagedControlPlane has a deletion timestamp, skipping mapping" objectMapper="cpTomc" gcpmanagedcontrolplane="default-l7fz79/cluster-gke-control-plane"
I0320 15:26:05.390819 1 gcpmanagedcluster_controller.go:229] "Reconciling Delete GCPManagedCluster" controller="gcpmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedCluster" GCPManagedCluster="default-l7fz79/cluster-gke" namespace="default-l7fz79" name="cluster-gke" reconcileID="ca724dd8-e462-47b6-b409-e8eb9b518da6" controller="gcpmanagedcluster" action="delete"
I0320 15:26:05.390885 1 gcpmanagedcluster_controller.go:232] "GCPManagedControlPlane not deleted yet, retry later" controller="gcpmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedCluster" GCPManagedCluster="default-l7fz79/cluster-gke" namespace="default-l7fz79" name="cluster-gke" reconcileID="ca724dd8-e462-47b6-b409-e8eb9b518da6" controller="gcpmanagedcluster" action="delete"
I0320 15:26:05.391158 1 gcpmanagedmachinepool_controller.go:321] "Reconciling GCPManagedMachinePool" controller="gcpmanagedmachinepool" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedMachinePool" GCPManagedMachinePool="default-l7fz79/cluster-gke-mp-0" namespace="default-l7fz79" name="cluster-gke-mp-0" reconcileID="c4234cfd-3c3b-452d-9d0c-3ede223e9619" controller="gcpmanagedmachinepool"
I0320 15:26:05.391419 1 gcpmanagedcontrolplane_controller.go:195] "Deleting GCPManagedControlPlane" controller="gcpmanagedcontrolplane" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedControlPlane" GCPManagedControlPlane="default-l7fz79/cluster-gke-control-plane" namespace="default-l7fz79" name="cluster-gke-control-plane" reconcileID="fea3b666-dbb7-41ba-81c9-85df5991b125" controller="gcpmanagedcontrolplane" action="delete"
I0320 15:26:05.391446 1 reconcile.go:184] "Deleting cluster resources" controller="gcpmanagedcontrolplane" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedControlPlane" GCPManagedControlPlane="default-l7fz79/cluster-gke-control-plane" namespace="default-l7fz79" name="cluster-gke-control-plane" reconcileID="fea3b666-dbb7-41ba-81c9-85df5991b125" service="container.clusters"
I0320 15:26:05.391565 1 reconcile.go:61] "Reconciling node pool resources" controller="gcpmanagedmachinepool" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedMachinePool" GCPManagedMachinePool="default-l7fz79/cluster-gke-mp-0" namespace="default-l7fz79" name="cluster-gke-mp-0" reconcileID="c4234cfd-3c3b-452d-9d0c-3ede223e9619"
I0320 15:26:05.488969 1 reconcile.go:201] "Cluster reconciling in progress" controller="gcpmanagedcontrolplane" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedControlPlane" GCPManagedControlPlane="default-l7fz79/cluster-gke-control-plane" namespace="default-l7fz79" name="cluster-gke-control-plane" reconcileID="fea3b666-dbb7-41ba-81c9-85df5991b125" service="container.clusters"
I0320 15:26:05.489902 1 controller.go:110] "Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference" controller="gcpmanagedcontrolplane" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="GCPManagedControlPlane" GCPManagedControlPlane="default-l7fz79/cluster-gke-control-plane" namespace="default-l7fz79" name="cluster-gke-control-plane" reconcileID="fea3b666-dbb7-41ba-81c9-85df5991b125"
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x28f94b5]
goroutine 353 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:111 +0x1e5
panic({0x305a0e0?, 0x5ae1120?})
runtime/panic.go:770 +0x132
sigs.k8s.io/cluster-api-provider-gcp/exp/controllers.(*GCPManagedControlPlaneReconciler).reconcileDelete(0xc0021ed8c0?, {0x3cef628, 0xc0005436c0}, 0xc002494500)
sigs.k8s.io/cluster-api-provider-gcp/exp/controllers/gcpmanagedcontrolplane_controller.go:218 +0x695
sigs.k8s.io/cluster-api-provider-gcp/exp/controllers.(*GCPManagedControlPlaneReconciler).Reconcile(0xc0000ca9f0, {0x3cef580?, 0xc00133f260?}, {{{0xc002068840?, 0x0?}, {0xc001484f40?, 0xc000b57d10?}}})
sigs.k8s.io/cluster-api-provider-gcp/exp/controllers/gcpmanagedcontrolplane_controller.go:148 +0x66d
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x3cf66e0?, {0x3cef580?, 0xc00133f260?}, {{{0xc002068840?, 0xb?}, {0xc001484f40?, 0x0?}}})
sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0009b6160, {0x3cef5b8, 0xc0008f8410}, {0x3188560, 0xc002051600})
sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 +0x3bc
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0009b6160, {0x3cef5b8, 0xc0008f8410})
sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 +0x1be
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 166
sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:218 +0x486
Eventually, the cluster is successfully deleted and the controller recovers from this panic but it should be fixed and handled properly.
What did you expect to happen:
The cluster deletes with no errors in the logs.
Anything else you would like to add:
Environment:
- Cluster-api version: v1.9
- Minikube/KIND version:
- Kubernetes version: (use
kubectl version): - OS (e.g. from
/etc/os-release):