Bug:GCU:speed or lane change takes much longer if the port already has ACL on it.

[rraghav-cisco]

Is it platform specific

generic

Importance or Severity

Critical

Description of the bug

I have 2 patches, acl and port_config:

root@yy40-mixed-short:~/gcu_patches# cat acl_only_patch2.json 
[
  {
    "op": "add",
    "path": "/asic2/ACL_TABLE/EVERFLOWV6/ports/0",
    "value": "Ethernet192"
  },
  {
    "op": "add",
    "path": "/asic2/ACL_TABLE/EVERFLOWV6/ports/1",
    "value": "Ethernet200"
  },
  {
    "op": "add",
    "path": "/asic2/ACL_TABLE/EVERFLOW/ports/0",
    "value": "Ethernet192"
  },
  {
    "op": "add",
    "path": "/asic2/ACL_TABLE/EVERFLOW/ports/1",
    "value": "Ethernet200"
  },
  {
    "op": "add",
    "path": "/asic2/ACL_TABLE/DATAACL/ports/0",
    "value": "Ethernet192"
  },
  {
    "op": "add",
    "path": "/asic2/ACL_TABLE/DATAACL/ports/1",
    "value": "Ethernet200"
  }
  
]
root@yy40-mixed-short:~/gcu_patches# 

root@yy40-mixed-short:~/gcu_patches# cat all_port_changes.json
[

  {
    "op": "add",
    "path": "/asic2/PORT/Ethernet192/fec",
    "value": "none"
  },
  {
    "op": "add",
    "path": "/asic2/PORT/Ethernet192/admin_status",
    "value": "up"
  },
  {
    "op": "replace",
    "path": "/asic2/PORT/Ethernet192/lanes",
    "value": "1288,1289,1290,1291"
  },
  {
    "op": "replace",
    "path": "/asic2/PORT/Ethernet192/description",
    "value": "Ixia-5.3:Ethernet1"
  },
  {
    "op": "replace",
    "path": "/asic2/PORT/Ethernet192/speed",
    "value": "100000"
  },
  {
    "op": "add",
    "path": "/asic2/PORT/Ethernet200/fec",
    "value": "none"
  },
  {
    "op": "add",
    "path": "/asic2/PORT/Ethernet200/admin_status",
    "value": "up"
  },
  {
    "op": "replace",
    "path": "/asic2/PORT/Ethernet200/lanes",
    "value": "1280,1281,1282,1283"   
  },
  {
    "op": "replace",
    "path": "/asic2/PORT/Ethernet200/description",
    "value": "Ixia-5.4:Ethernet1"
  },
  {
    "op": "replace",
    "path": "/asic2/PORT/Ethernet200/speed",
    "value": "100000"
  },
  {
    "op": "add",
    "path": "/asic2/DEVICE_NEIGHBOR/Ethernet200",
    "value": {
      "name": "Ixia-5.4",
      "port": "Ethernet1"
    }
  },
  {
    "op": "add",
    "path": "/asic2/DEVICE_NEIGHBOR/Ethernet192",
    "value": {
      "name": "Ixia-5.3",
      "port": "Ethernet1"
    }
  },
  {
    "op": "add",
    "path": "/asic2/INTERFACE/Ethernet200",
    "value": {}
  },
  {
    "op": "add",
    "path": "/asic2/INTERFACE/Ethernet200|FC00:123::1~1126",
    "value": {}
  },
  {
    "op": "add",
    "path": "/asic2/INTERFACE/Ethernet192",
    "value": {}
  },
  {
    "op": "add",
    "path": "/asic2/INTERFACE/Ethernet200|10.1.123.1~124",
    "value": {}
  },
  {
    "op": "add",
    "path": "/asic2/INTERFACE/Ethernet192|10.1.122.1~124",
    "value": {}
  },
  {
    "op": "add",
    "path": "/asic2/INTERFACE/Ethernet192|FC00:122::1~1126",
    "value": {}
  }
]

When the patches are applied independently after a config reload, they take very little time(<30 seconds).
Each translates to about 6 and 30 changes.

However, if I apply acl and then port_config, the port_config takes about 30 minutes. The port config translates to 600 changes, which also includes remove operations for the acls, and add operations again.

@abdosi , pls let me know if this is expected.

Steps to Reproduce

Apply the above set of patches.

Actual Behavior and Expected Behavior

I am not sure if this is expected.

Relevant log output

Output of show version, show techsupport

Attach files (if any)

No response

[anamehra]

Hi @rraghav-cisco , The ACL rules should be present only for ports which are already programmed. For new ports being configured via GCU, I don't think ACL rules will be present on the card. The new patch should add new port in 100G. This port will be present in admin down with 400G and no other config.

When adding new config, can we apply port add + qos and then add acl config for new ports?

[prgeor]

@bhouse-nexthop offered help to take a look into this issue

[bhouse-nexthop]

i have a fix for this coming. The removal of all ACLs is the exact symptom we see. The underlying cause is RemoveCreateOnlyDependencyMoveValidator just not doing things right at all. When you change the lanes member of a Port, it actually has to remove the port completely and re-add it. To do that, it must remove all dependencies, remove the port, add the port back with new settings, then restore any dependencies.

We are just finishing up internal QA and review on this and should post the PR in the next day or two. I will link to this ticket.

[bhouse-nexthop]

Likely this is fixed by sonic-net/sonic-utilities#4335 ... If not, please run config apply-patch with --path-trace /tmp/trace.json and attach the resulting output. (NOTE: that trace option was added in sonic-net/sonic-utilities#4317)