The setMirrorRoute: step does not support Port match

[zhongdai]

Here is the background,

I have a service accepts both REST and gRPC requests on different port, and I am using host level canary rollout, so my virtual service looks like the below,

• the REST is on 8000, and the gRPC is on 9000
• when there is no canary, the weight for the stable destination is 100%

http:
- match:
  - port: 8000
    uri:
      prefix: /
  name: rest-api-routes
  route:
  - destination:
      host: my-service
      port:
        number: 8000
    weight: 100
  - destination:
      host: my-service-canary
      port:
        number: 8000
    weight: 0
- match:
  - port: 9000
    uri:
      prefix: /
  name: grpc-api-routes
  route:
  - destination:
      host: my-service
      port:
        number: 9000
    weight: 100
  - destination:
      host: my-service-canary
      port:
        number: 9000
    weight: 0

Now I plan to mirror 50% traffic to the canary replica set, so here is my rollout canary steps,

canary:
  canaryService: my-service-canary
  stableService: my-service
  stableMetadata:
    labels:
      role: stable
  canaryMetadata:
    labels:
      role: canary
  trafficRouting:
    managedRoutes:
      - name: mirror-route
    istio:
      virtualService:
        name: the-virtual-service-name
        routes:
          - rest-api-routes # <- this one will be pick to make the mirror-route
          - grpc-api-routes
  steps:
    - setCanaryScale:
        weight: 100
    - setMirrorRoute:
        name: mirror-route
        percentage: 50
        match: # based on the document, I cannot match by port??
          - path:
              prefix: /
    - pause: { 20m } # warm up time
    - setMirrorRoute:
        name: mirror-route # removes mirror based traffic route
    - setWeight: 100

It seem the Argo Rollout picks the rest-api-routes and adding the mirror and mirrorPercentage sections, then prepend it to my virtual service during the rollout, so there is a new http route looks like

- match:
  - uri:
      prefix: /
  mirror:
    host: my-service-canary
    port:
      number: 8000
  mirrorPercentage:
    value: 50
  name: mirror-route
  route:
  - destination:
      host: my-service
      port:
        number: 8000
    weight: 100
  - destination:
      host: my-service-canary
      port:
        number: 8000

This cause some critical problems,

1 - I lost all my gRPC requests, the newly added route just forward everything to my stable service regardless the port match, including the gRPC requests, and they won't be processed, because the port 8000 is for the REST
2 - use grpc-api-routes or rest-api-routes depends on the order I set in the Canary section, that is very undertimstic

I am thinking to add mirror-route-1 and mirror-route-2, but I don't think that would work, because I cannot specify the Port matching for the setMirrorRoute step.

Appreciate for any information, thoughts or suggestion.

[RunlinXiao]

I'm running into the same issue and it wasn't easy to debug, I've opened an enhancement proposal to address the root cause of this issue: #4481