Skip to content

Skip primary check for appmesh#315

Merged
stefanprodan merged 1 commit intofluxcd:masterfrom
nil-scan:appmesh-init
Oct 2, 2019
Merged

Skip primary check for appmesh#315
stefanprodan merged 1 commit intofluxcd:masterfrom
nil-scan:appmesh-init

Conversation

@nil-scan
Copy link
Contributor

@nil-scan nil-scan commented Oct 2, 2019

Not sure if this is the right way to fix it but it worked for me (I'm not familiar with Istio but I suspect that the exception in the code has been added for similar reasons).

The problem is that when creating a new canary resource, Flagger waits for the primary pod to be ready before creating the Appmesh VirtualNode. If the deployment has an http readyness probe configured, it will always fail because Envoy can't get its config (as the VirtualNode doesn't exist yet) and rejects all connections.

We basically end up in the following state:

NAME                                        READY   STATUS             RESTARTS   AGE
test-canary-tuto-66cd89fd7b-98dnf           1/2     CrashLoopBackOff   6          4m39s
test-canary-tuto-66cd89fd7b-df9w7           1/2     CrashLoopBackOff   6          4m39s
test-canary-tuto-primary-597f6bf797-5jpfp   1/2     CrashLoopBackOff            1          47s
test-canary-tuto-primary-597f6bf797-q2995   1/2     CrashLoopBackOff            1          47s

And flagger the flagger logs shows a bunch of:

{"level":"info","ts":"2019-10-02T14:26:50.855+1300","caller":"controller/controller.go:271","msg":"Halt advancement test-canary-tuto-primary.test waiting for rollout to finish: 0 of 2 updated replicas are available","canary":"test-canary-tuto.test"}

@nil-scan nil-scan requested a review from stefanprodan as a code owner October 2, 2019 01:36
Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks @nilscan

@stefanprodan stefanprodan merged commit e86f627 into fluxcd:master Oct 2, 2019
@stefanprodan stefanprodan mentioned this pull request Oct 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants