Header based routing fails with managed route rules removed immediately if there's no canary pod scale step prior to setHeaderRoute

[baburciu]

Checklist:

• I've included steps to reproduce the bug.
• I've included the version of argo rollouts.
• I've included the version of the gateway API plugin
• I've included the name and version of the gateway API provider I am using
• I've included my Rollout Spec (anonymized if needed)

Describe the bug

Header based routing in GatewayAPI plugin fails for us because the RemoveManagedRoutes function is immediately removing header routes after they are created (i.e. dropping both HTTPRoute.spec.rules entry and ConfigMap/argo-gatewayapi-configmap httpManagedRoutes contents). AFAIU, plugin RemoveManagedRoutes is over-aggressive, it removes routes during normal canary progression instead of only during completion/abort.
This is evident from:

kubectl get configmap argo-gatewayapi-configmap -n dev -o yaml --watch --show-managed-fields=true (click to expand)

$ kubectl get configmap argo-gatewayapi-configmap -n dev -o yaml --watch --show-managed-fields=true
apiVersion: v1
data:
  httpManagedRoutes: '{}'
kind: ConfigMap
metadata:
  creationTimestamp: "2025-09-30T15:44:57Z"
  name: argo-gatewayapi-configmap
  namespace: dev
  resourceVersion: "8586412"
  uid: 7c7b593c-02ba-40de-9835-66147a332f3f
---
apiVersion: v1
data:
  httpManagedRoutes: '{"demo-application-canary-header":{"demo-application":1}}'
kind: ConfigMap
metadata:
  creationTimestamp: "2025-09-30T15:44:57Z"
  name: argo-gatewayapi-configmap
  namespace: dev
  resourceVersion: "8613276"
  uid: 7c7b593c-02ba-40de-9835-66147a332f3f
---
apiVersion: v1
data:
  httpManagedRoutes: '{}'
kind: ConfigMap
metadata:
  creationTimestamp: "2025-09-30T15:44:57Z"
  name: argo-gatewayapi-configmap
  namespace: dev
  resourceVersion: "8613316"
  uid: 7c7b593c-02ba-40de-9835-66147a332f3f

kubectl get httproute demo-application -n dev -o yaml --watch --show-managed-fields=true (click to expand)

$ kubectl get httproute demo-application -n dev -o yaml --watch --show-managed-fields=true
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 9
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:47:38Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:47:38Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8591161"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:47:38Z"
      message: Accepted HTTPRoute
      observedGeneration: 9
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:47:38Z"
      message: Service reference is valid
      observedGeneration: 9
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 9
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
        f:rules: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:47:38Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:47:38Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8613237"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:47:38Z"
      message: Accepted HTTPRoute
      observedGeneration: 9
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:47:38Z"
      message: Service reference is valid
      observedGeneration: 9
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 10
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:47:38Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:26Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8613274"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 1
    matches:
    - headers:
      - name: X-Canary
        type: Exact
        value: "yes"
      path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:47:38Z"
      message: Accepted HTTPRoute
      observedGeneration: 9
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:47:38Z"
      message: Service reference is valid
      observedGeneration: 9
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 10
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:52:26Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:26Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8613278"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 1
    matches:
    - headers:
      - name: X-Canary
        type: Exact
        value: "yes"
      path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Accepted HTTPRoute
      observedGeneration: 10
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Service reference is valid
      observedGeneration: 10
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 11
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:52:26Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:26Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8613281"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - headers:
      - name: X-Canary
        type: Exact
        value: "yes"
      path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Accepted HTTPRoute
      observedGeneration: 10
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Service reference is valid
      observedGeneration: 10
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 11
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:52:26Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:26Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8613287"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - headers:
      - name: X-Canary
        type: Exact
        value: "yes"
      path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Accepted HTTPRoute
      observedGeneration: 11
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Service reference is valid
      observedGeneration: 11
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 12
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:52:26Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:26Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8613314"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Accepted HTTPRoute
      observedGeneration: 11
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Service reference is valid
      observedGeneration: 11
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 12
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:52:26Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:26Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8613317"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 100
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 0
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Accepted HTTPRoute
      observedGeneration: 12
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Service reference is valid
      observedGeneration: 12
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 13
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:52:26Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:47Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8615056"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 50
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 50
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Accepted HTTPRoute
      observedGeneration: 12
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:52:26Z"
      message: Service reference is valid
      observedGeneration: 12
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"
    argocd.argoproj.io/tracking-id: demo-application-dev:gateway.networking.k8s.io/HTTPRoute:dev/demo-application
  creationTimestamp: "2025-09-30T15:44:54Z"
  generation: 13
  labels:
    app.kubernetes.io/instance: demo-application-dev
    app.kubernetes.io/managed-by: Helm
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:argocd.argoproj.io/sync-wave: {}
          f:argocd.argoproj.io/tracking-id: {}
        f:labels:
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:helm.sh/chart: {}
      f:spec:
        f:hostnames: {}
        f:parentRefs: {}
    manager: argocd-controller
    operation: Apply
    time: "2025-09-30T15:44:54Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:parents: {}
    manager: cilium-operator-aws
    operation: Update
    subresource: status
    time: "2025-09-30T15:52:47Z"
  - apiVersion: gateway.networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:rules: {}
    manager: gatewayAPI
    operation: Update
    time: "2025-09-30T15:52:47Z"
  name: demo-application
  namespace: dev
  resourceVersion: "8615060"
  uid: 90b05317-93a4-45a7-84c7-c6cbb41d132d
spec:
  hostnames:
  - dummy
  parentRefs:
  - group: ""
    kind: Service
    name: demo-application
    namespace: dev
    port: 80
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: demo-application-stable
      port: 80
      weight: 50
    - group: ""
      kind: Service
      name: demo-application-canary
      port: 80
      weight: 50
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-09-30T15:52:47Z"
      message: Accepted HTTPRoute
      observedGeneration: 13
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-09-30T15:52:47Z"
      message: Service reference is valid
      observedGeneration: 13
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: io.cilium/gateway-controller
    parentRef:
      group: ""
      kind: Service
      name: demo-application
      namespace: dev
      port: 80

watch-rollout.md

argo-rollouts-controller-logs.md

To Reproduce

Use:

• argo-rollouts helm chart version 2.40.4, app version v1.8.3
• with helm value

  trafficRouterPlugins:
    - location: https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/v0.8.0/gatewayapi-plugin-linux-amd64
      name: argoproj-labs/gatewayAPI

• and cilium helm chart version 1.18.2, with Gateway API v1.2.0 CRDs, as GatewayAPI provider
• in an EKS cluster (Cilium as only CNI, no AWS VPC CNI chaining)

Deploy a canary Rollout like this:

# $ kubectl get rollout demo-application-dev-api -n dev -o yaml | yq .spec.strategy
canary:
  analysis:
    analysisRunMetadata: {}
    startingStep: 2
    templates:
      - templateName: demo-application-dev-api-analysis-template
  canaryService: demo-application-canary
  stableService: demo-application-stable
  steps:
    - setHeaderRoute:
        match:
          - headerName: X-Canary
            headerValue:
              exact: "yes"
        name: demo-application-canary-header
    - setWeight: 50
    - pause:
        duration: 50m
    - setWeight: 100
  trafficRouting:
    managedRoutes:
      - name: demo-application-canary-header
    plugins:
      argoproj-labs/gatewayAPI:
        httpRoutes:
          - name: demo-application
            useHeaderRoutes: true
        namespace: dev

managing traffic for a GAMMA producer HTTPRoute:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: demo-application
spec:
  hostnames:
  - dummy  # does not count in GAMMA GatewayAPI yet, see https://github.com/kubernetes-sigs/gateway-api/issues/2885
  parentRefs:
    - group: ""
      kind: Service
      name: demo-application
      port: 80
      namespace: dev
  rules:
    - backendRefs:
        - group: ''
          kind: Service
          name: demo-application-stable
          port: 80
          weight: 100
        - group: ''
          kind: Service
          name: demo-application-canary
          port: 80
          weight: 0

similar to the LinkerD example.

Expected behavior

I would expect that during Rollout (pause) step 2 my HTTPRoute to have a rule programmed for Header Based Routing. As it can be seen in the above kubectl output, that is not the case. The rule is there but for a very short time. It should be there until Rollout completes.

Observed behavior has curl -H "X-Canary: yes" http://demo-application/<route> return results from both stable and canary pods/services.

I'd expect to have the HTTPRoute contain a canary header matching rule until the Rollout finishes, but it was not the case in my tests (please see above kubectl get httproute demo-application -n dev -o yaml --watch --show-managed-fields=true output).

Screenshots

Version

v1.8.3

Logs
Please find controller logs attached. A single Rollout was managed by the controller.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[baburciu]

This seems to be related to the usage of setHeaderRoute as first canary Rollout step, with:

      steps:
        - setHeaderRoute:
            match:
              - headerName: X-Test
                headerValue:
                  exact: test
            name: argo-rollouts
        - setWeight: 50
        - pause:
            duration: 50m
        - setWeight: 100
        - setHeaderRoute:
            name: argo-rollouts

I can see the same issue (canary header matching rule added and immediately deleted from GAMMA HTTPRoute) experienced with both Cilium and Linkerd GatewayAPI implementations. It can be reproduced following:

• Cilium example
• Linkerd example

[baburciu]

It seems to work if a canary pod scaling step is done before setHeaderRoute, for example:

      steps:
        - setCanaryScale:
            weight: 20
        - setHeaderRoute:
            match:
              - headerName: X-Test
                headerValue:
                  exact: test
            name: argo-rollouts
        - setWeight: 50
        - pause:
            duration: 50m
        - setWeight: 100
        - setHeaderRoute:
            name: argo-rollouts

allows for the HTTPRoute rule to stick