Skip to content

Gateway infrastructure.annotations value changes not propagated to managed Service on update #4822

Description

@raakash121

Describe the bug

When infrastructure.annotations values are updated in the Gateway spec (e.g. changing the value of service.beta.kubernetes.io/aws-load-balancer-attributes), NGINX Gateway Fabric does not propagate the updated annotation value to the managed Service. The annotation key is tracked correctly in gateway.nginx.org/internal-managed-annotation-keys but the Service retains the old value even after helm upgrade or direct Gateway annotation update.

To Reproduce

  1. Deploy NGINX Gateway Fabric with infrastructure.annotations containing:
service.beta.kubernetes.io/aws-load-balancer-attributes: "deletion_protection.enabled=true"
  1. Update the annotation value in Gateway spec to:
service.beta.kubernetes.io/aws-load-balancer-attributes: "deletion_protection.enabled=true,load_balancing.cross_zone.enabled=true"
  1. Run helm upgrade with the updated values
  2. Check the managed Service annotations:
kubectl get svc <nginx-service> -n <namespace> -o yaml | grep aws-load-balancer-attributes
  1. Observe that the Service still has the old value deletion_protection.enabled=true despite the Gateway having the new value

Expected behavior

When infrastructure.annotations values are updated on the Gateway object, NGF should detect the value change and propagate the updated annotation value to the managed Service automatically — without requiring manual Service deletion and recreation.

Your environment

  • Version of NGINX Gateway Fabric: 2.4.1
  • Version of Kubernetes: EKS (AWS) : v1.31
  • Kubernetes platform: AWS EKS
  • NGINX Gateway Fabric exposed via: Service of type LoadBalancer managed by AWS Load Balancer Controller
  • Logs of NGINX container:

**Additional context**

- The annotation key IS correctly tracked in `gateway.nginx.org/internal-managed-annotation-keys` on the Service
- The Gateway object reflects the correct updated annotation value
- Only the Service annotation value is stale
- Workaround: Manually annotate the Service after helm upgrade, or delete and recreate the Service to force NGF to reconcile with the latest Gateway annotation values
- This is particularly impactful when using AWS Load Balancer Controller, where Service annotations control NLB attributes — stale annotations mean NLB attributes are not updated

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcommunityrefinedRequirements are refined and the issue is ready to be implemented.waiting for responseWaiting for author's response

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    ✅ Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions