v0.12.1 | Controller Error "pods already exists" causing reconciliation error to spike

[rajesh-dhakad]

Checks

• I've already read https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/troubleshooting-actions-runner-controller-errors and I'm sure my issue is not covered in the troubleshooting guide.
• I am using charts that are officially provided

Controller Version

0.12.1

Deployment Method

Helm

Checks

• This isn't a question or user support case (For Q&A and community support, go to Discussions).
• I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

None

Describe the bug

Seeing a spike in reconciliation error.

Describe the expected behavior

The controller should handle this error.

Additional Context

None

Controller Logs

{"severity":"error","ts":"2025-08-21T12:04:10Z","logger":"EphemeralRunner","message":"Failed to create pod resource for ephemeral runner.","version":"0.12.1","ephemeralrunner":{"name":"general-low-6vl9g-runner-wzh24","namespace":"arc-runners"},"error":"pods \"general-low-6vl9g-runner-wzh24\" already exists","stacktrace":"github.com/actions/actions-runner-controller/controllers/actions%2egithub%2ecom.(*EphemeralRunnerReconciler).createPod\n\tgithub.com/actions/actions-runner-controller/controllers/actions.github.com/ephemeralrunner_controller.go:663\ngithub.com/actions/actions-runner-controller/controllers/actions%2egithub%2ecom.(*EphemeralRunnerReconciler).Reconcile\n\tgithub.com/actions/actions-runner-controller/controllers/actions.github.com/ephemeralrunner_controller.go:246\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:334\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:255"}
{"severity":"error","ts":"2025-08-21T12:04:10Z","logger":"EphemeralRunner","message":"Failed to create the pod","version":"0.12.1","ephemeralrunner":{"name":"general-low-6vl9g-runner-wzh24","namespace":"arc-runners"},"error":"pods \"general-low-6vl9g-runner-wzh24\" already exists","stacktrace":"github.com/actions/actions-runner-controller/controllers/actions%2egithub%2ecom.(*EphemeralRunnerReconciler).Reconcile\n\tgithub.com/actions/actions-runner-controller/controllers/actions.github.com/ephemeralrunner_controller.go:259\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:334\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:255"}
{"severity":"error","ts":"2025-08-21T12:04:10Z","message":"Reconciler error","controller":"ephemeralrunner","controllerGroup":"actions.github.com","controllerKind":"EphemeralRunner","EphemeralRunner":{"name":"general-low-6vl9g-runner-wzh24","namespace":"arc-runners"},"namespace":"arc-runners","name":"general-low-6vl9g-runner-wzh24","reconcileID":"af2982c8-a3de-4f25-9299-bcdec9658ead","error":"pods \"general-low-6vl9g-runner-wzh24\" already exists","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:347\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.20.4/pkg/internal/controller/controller.go:255"}

Runner Pod Logs

None

[rajesh-dhakad]

@nikola-jokic : My bad, just saw this PR which handles the error - #4201

But I don't see any new release of the controller that should have these updates included?

[nikola-jokic]

Hey @rajesh-dhakad,

Yes, we should close this issue for now, but just to make sure we understand this error, the root cause is that the controller didn't get the pod in a timely manner.

The reconciliation loop creates the pod. We have a watcher that should deliver that pod to the controller. Next time we reconcile, due to a delay, we will try to re-create the same pod, causing this error. The error itself is not causing delays; something else on the cluster is delaying delivery of the pod. Therefore, to properly fix the issue, you should investigate what is causing delivery to be slow.