Skip to content

Conversation

@artem-zinnatullin
Copy link
Contributor

We've started experiencing issues with K8S cluster (unrelated to k8s-buildkite-plugin) and they slip in as kubectl errors in the middle of our build logs on Buildkite like this:

Uploading: /tmp/tmp.N7WoODSLLy/dist/symbols/armeabi-v7a.objdump.gz for arch: armeabi-v7a with sharedObjectName:gl.so to https://upload.bugsnag.com/so-symbol
--
  | error: You must be logged in to the server (Unauthorized)
  | error: You must be logged in to the server (Unauthorized)
  | error: You must be logged in to the server (Unauthorized)
  | error: You must be logged in to the server (Unauthorized)
  | NDK mapping NdkMappings(sharedObjectName=x.so, arch=arm64-v8a, ndkMappingFilePath=/tmp/tmp.N7WoODSLLy/capture_symbols/arm64-v8a.objdump.gz), NdkMappings(sharedObjectName=x.so, arch=armeabi-v7a, ndkMappingFilePath=/tmp/tmp.N7WoODSLLy/capture_symbols/armeabi-v7a.objdump.gz) has been uploaded successfully!
  | ProGuard mapping /tmp/tmp.X9Eev60Pxz has been uploaded successfully!
  | The operation took 27.154 sec.

This is due to k8s-buildkite-plugin outputting logs in parallel to the while true control loops where we check job/pod status, etc and ANY kubectl command can fail and print an error that doesn't at all indicate that it is coming from kubectl 🙃

PR wraps all kubectl calls, captures stderr output and prints it with the kubectl command parameters so that it is very clear

  1. kubectl failed
  2. What kubectl command failed (I've checked the invocations, parameter values don't seem to include any sensitive info like tokens)

@artem-zinnatullin
Copy link
Contributor Author

Friendly ping @tgolsson 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant