Worker + k8s runtime panic in InspectContainer #564
Description
Description
With the kubernetes runtime:
When a build terminates early (eg a step failed), many of the containers will still be "running" with the kubernetes/pause:latest image. But InspectContainer assumes it is getting called after the container has terminated. When it accesses the termination details, it causes a panic:
vela-worker-6bd9b8587b-sntzs worker panic: runtime error: invalid memory address or nil pointer dereference
vela-worker-6bd9b8587b-sntzs worker [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x15025fd]
vela-worker-6bd9b8587b-sntzs worker
vela-worker-6bd9b8587b-sntzs worker goroutine 14 [running]:
vela-worker-6bd9b8587b-sntzs worker github.com/go-vela/worker/runtime/kubernetes.(*client).InspectContainer(0xc00059a960, {0xc000000005, 0xc00090b3c8}, 0xc00003fb00)
vela-worker-6bd9b8587b-sntzs worker /vela/runtime/kubernetes/container.go:58 +0x21d
vela-worker-6bd9b8587b-sntzs worker github.com/go-vela/worker/executor/linux.(*client).DestroyStep(0xc0005aa780, {0x1d2de88, 0xc00004a030}, 0xc00003fb00)
vela-worker-6bd9b8587b-sntzs worker /vela/executor/linux/step.go:441 +0x2c3
vela-worker-6bd9b8587b-sntzs worker github.com/go-vela/worker/executor/linux.(*client).DestroyStage(0xc0005aa780, {0x1d2de88, 0xc00004a030}, 0xc0000925a0)
vela-worker-6bd9b8587b-sntzs worker /vela/executor/linux/stage.go:161 +0x20c
vela-worker-6bd9b8587b-sntzs worker github.com/go-vela/worker/executor/linux.(*client).DestroyBuild(0xc0005aa780, {0x1d2de88, 0xc00004a030})
vela-worker-6bd9b8587b-sntzs worker /vela/executor/linux/build.go:531 +0x325
vela-worker-6bd9b8587b-sntzs worker main.(*Worker).exec(0xc000387140, 0x0)
vela-worker-6bd9b8587b-sntzs worker /vela/cmd/vela-worker/exec.go:147 +0xdf5
vela-worker-6bd9b8587b-sntzs worker main.(*Worker).operate.func2()
vela-worker-6bd9b8587b-sntzs worker /vela/cmd/vela-worker/operate.go:109 +0x89
vela-worker-6bd9b8587b-sntzs worker golang.org/x/sync/errgroup.(*Group).Go.func1()
vela-worker-6bd9b8587b-sntzs worker /go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57 +0x67
vela-worker-6bd9b8587b-sntzs worker created by golang.org/x/sync/errgroup.(*Group).Go
vela-worker-6bd9b8587b-sntzs worker /go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:54 +0x92
Value
no panic: good
panic: bad
/me likes no panic better
Useful Information
- What is the output of
vela --version?
worker is on the master branch
- What operating system is being used?
linux
- Any other important details?
fix forthcoming. I'm just trying to file more issues before I push the PRs.