How do enforce argo-rollouts to do full-promote on every upcoming deployment?

[ardikabs]

We have a setup with two clusters: a primary and a secondary. The secondary cluster is intended to remain dormant and only be activated during a disaster or under specific circumstances.

Our services are deployed using Argo CD, referencing a single source of truth for the manifests. However, when we bring up the secondary cluster, we’ve observed that Argo Rollouts retains the previous rollout state and performs a canary deployment if the manifests have changed. This behavior is not ideal for our use case, when the secondary cluster is activated, we need all services to immediately match the latest version running in the primary cluster.

Our current workaround is to manually trigger a full promotion for all services in the secondary cluster.

That leads to our question: is it possible to configure Argo Rollouts to perform a full promotion by default, instead of starting with a canary deployment?

As a note, this has become a critical issue for us. During the preparation of the secondary cluster, services with updated images may run a mix of versions; the latest image (as defined in the primary cluster) and an older version (from the last rollout state). This can lead to problematic behavior, as certain background tasks may be executed by the outdated version, potentially causing harmful outcomes.

Thanks

[ardikabs]

I'm aware of the following related issues:

• Centralized Rollout for multi-cluster environments #2737
• Support multi-rollout promotion process #2088

However, my question is more about whether there's an option that could ideally configurable within the Argo Rollouts controller to toggle whether the full-promote mechanism should be enabled by default. If such a feature already exists, I’d appreciate any guidance. If not, and it's something that could potentially be introduced, I'd be more than happy to contribute toward its implementation.

[kostis-codefresh]

A quick workaround would be to have whatever process launches the second cluster to also do a kubectl argo rollouts promote.

However I fail to understand exactly your use case? How does Argo Rollout keep previous state. Isn't the second cluster brand new? Or do you backup regularly the first cluster to the second.

How does a canary process start in the first place? If primary cluster has my-app:v1.2 as a rollout, doesn't the backup cluster also have my-app:v1.2 as a rollout as well?

[ardikabs]

hi @kostis-codefresh, that's exactly part of the problem, as unfortunately our second cluster is not really cluster brand new, as because we have a periodic testing our what so-called a DR cluster, hence the previous state remain.

Currently, our deployment strategy uses the same manifest reference in ArgoCD. Initially, we configured auto-sync to the secondary cluster. However, because old states persist, each time we bring up the DR cluster we need to handle the Rollout object. To work around this, instead of allowing rollouts to run, we simply manually delete them before starting the DR spin-up.

Ideally, it would be more efficient if the argo-rollouts-controller had a default “full promote” option, so that once it starts, it would immediately override the last state and skip the canary steps.