Trying to debug why notification attempt is complaining about secret

[davidmichaelkarr]

Using Rollouts v1.8.3+49fa151 , and argo cd v2.14.15+6c7d694 (note recent version of rollouts, older version of argo cd).

I have a simple Argo Rolllouts POC working along with ArgoCD. I'm now trying to implement notifications, as I'm going to have to integrate github workflows into the process. In the log, I see the following:

"config referenced 'token $GITHUB_TOKEN', but key does not exist in secret"

I need to know exactly what this means.

This is an excerpt from the configmap that I added:

data:
  service.webhook.github: |
    url: https://api.github.com/...
    headers:
      Authorization: token $GITHUB_TOKEN
  trigger.on-rollout-paused: |
    - send: [rollout-paused]
  template.rollout-paused: |
    message: |
     ...
  env: |
    - name: GITHUB_TOKEN
      valueFrom:
        secretKeyRef:
          name: ...
          key: password

I would assume that the error message means that the "key" referenced in the "secretKeyRef" does not exist in the given secret object.

Here is an excerpt from that secret object:

stringData:
  type: git
  url: https://github.com/...
  proxy: ...
  username: ...
  password: ...

As far as I can see, I do have a "password" property in the data block of the secret.

I later saw this: https://argo-cd.readthedocs.io/en/stable/operator-manual/notifications/troubleshooting/#config-referenced-xxx-but-key-does-not-exist-in-secret .

I wasn't aware of the need to have the "app.kubernetes.io/part-of: argocd" label. I added that. It also says "If you are using a custom secret, check that the secret is in the same namespace". Curiously, it doesn't say "the same namespace as ...". In any case, the configmap and the secret are both in the "argo-rollouts" namespace. I deleted the running controller pod and watched the new one start. I reran my test, same error. I even looked at the code that is likely emitting this, at https://github.com/argoproj/notifications-engine/blob/c449507755ab9594b96b50b6e7566f819cce7d67/pkg/api/config.go#L67 , but I don't see any clues from that.

Also note that right after that initial error message, I also see this:

error getting api from namespace: argo-rollouts error: error unmarshaling JSON: while decoding JSON: json: cannot unmarshal object into Go struct field WebhookOptions.headers of type []services.Header
notifications failed to get apis for eventReason RolloutAddedToInformer with error: errors getting apis: [error unmarshaling JSON: while decoding JSON: json: cannot unmarshal object into Go struct field WebhookOptions.headers of type []services.Header]

[davidmichaelkarr]

I have apparently resolved this particular problem. The way to specify the secret to use requires a completely different syntax. It appears to be getting past this, but I'm now getting an error about my trigger not being defined. I will post a separate question about that.