Fix buffer queue cnt test to account for different BUFFER_QUEUE configs

[zbud-msft]

Description of PR

Summary:
Fixes #19201
MSADO: 33479698

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• New Test case
  • Skipped for non-supported platforms
• Test case improvement

Back port request

• 202205
• 202305
• 202311
• 202405
• 202411
• 202505

Approach

What is the motivation for this PR?

Currently this test does not account for BUFFER_QUEUE configs to have only entry for an interface.

In some SKU's and config

BUFFER_QUEUE may have config such as

"BUFFER_QUEUE": {
    "Ethernet0|0-2": {
        "profile": "egress_lossy_profile"
    },      
    "Ethernet0|3-4": {
        "profile": "egress_lossless_profile"
    },      
    "Ethernet0|5-7": {
        "profile": "egress_lossy_profile"
    }, 

or even

"BUFFER_QUEUE": {
    "Ethernet0|0": {
        "profile": "egress_lossy_profile"
    },      
    "Ethernet0|1": {
        "profile": "egress_lossless_profile"
    },      
    "Ethernet0|2": {
        "profile": "egress_lossy_profile"
    },  ... etc

The test works fine for the above configs.

However if the BUFFER_QUEUE was configured such as

"BUFFER_QUEUE": {
    "Ethernet0|0-9": {
        "profile": "egress_lossy_profile"
    },      
    "Ethernet100|0-9": {
        "profile": "egress_lossless_profile"
    },      
    "Ethernet200|0-9": {
        "profile": "egress_lossy_profile"
    },  

When we delete Ethernet0|0-9 as part of del data['BUFFER_QUEUE'][ interface_buffer_queues[0] ] we end up deleting all queues for that interface from COUNTERS_QUEUE_NAME_MAP which causes check_buffer_queues_cnt_cmd_output to always return false.

This test checks for such edge case and we split the BUFFER_QUEUE config such that we can delete an entry without deleting all entries in the map.

How did you do it?

Code change

How did you verify/test it?

Test on 20241110

01/07/2025 19:00:48 base._run                                L0108 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell Result => {"failed": true, "changed": true, "end": "2025-07-01 19:00:48.806079", "stdout": "Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path\n ------------------------- \n [elem {\n  name: \"COUNTERS_QUEUE_NAME_MAP\"\n}\nelem {\n  name: \"Ethernet0:0\"\n}\n]\nGRPC error\n redis: nil", "cmd": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:0             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "delta": "0:00:00.199525", "stderr": "", "rc": 1, "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:0             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "removes": null, "argv": null, "warn": true, "chdir": null, "stdin_add_newline": true, "stdin": null}}, "start": "2025-07-01 19:00:48.606554", "msg": "non-zero return code", "stdout_lines": ["Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path", " ------------------------- ", " [elem {", "  name: \"COUNTERS_QUEUE_NAME_MAP\"", "}", "elem {", "  name: \"Ethernet0:0\"", "}", "]", "GRPC error", " redis: nil"], "stderr_lines": [], "_ansible_no_log": false}
01/07/2025 19:00:48 base._run                                L0071 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell, args=["python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:1             -xt COUNTERS_DB -o \"xxxxxxxxx\"             "], kwargs={"module_ignore_errors": true}                                                                                                                                                                                                                                                                                                                          01/07/2025 19:00:49 base._run                                L0108 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell Result => {"changed": true, "end": "2025-07-01 19:00:49.227872", "stdout": "Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path\n ------------------------- \n [elem {\n  name: \"COUNTERS_QUEUE_NAME_MAP\"\n}\nelem {\n  name: \"Ethernet0:1\"\n}\n]\nThe GetResponse is below\n-------------------------\n\noid:0x15000000000679", "cmd": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:1             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "rc": 0, "start": "2025-07-01 19:00:48.981566", "stderr": "", "delta": "0:00:00.246306", "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:1             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "removes": null, "argv": null, "warn": true, "chdir": null, "stdin_add_newline": true, "stdin": null}}, "stdout_lines": ["Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path", " ------------------------- ", " [elem {", "  name: \"COUNTERS_QUEUE_NAME_MAP\"", "}", "elem {", "  name: \"Ethernet0:1\"", "}", "]", "The GetResponse is below", "-------------------------", "", "oid:0x15000000000679"], "stderr_lines": [], "_ansible_no_log": false, "failed": false}
01/07/2025 19:00:49 base._run                                L0071 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell, args=["python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:2             -xt COUNTERS_DB -o \"xxxxxxxxx\"             "], kwargs={"module_ignore_errors": true}                                                                                                                                                                                                                                                                                                                          01/07/2025 19:00:49 base._run                                L0108 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell Result => {"changed": true, "end": "2025-07-01 19:00:49.614484", "stdout": "Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path\n ------------------------- \n [elem {\n  name: \"COUNTERS_QUEUE_NAME_MAP\"\n}\nelem {\n  name: \"Ethernet0:2\"\n}\n]\nThe GetResponse is below\n-------------------------\n\noid:0x1500000000067a", "cmd": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:2             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "rc": 0, "start": "2025-07-01 19:00:49.411463", "stderr": "", "delta": "0:00:00.203021", "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:2             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "removes": null, "argv": null, "warn": true, "chdir": null, "stdin_add_newline": true, "stdin": null}}, "stdout_lines": ["Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path", " ------------------------- ", " [elem {", "  name: \"COUNTERS_QUEUE_NAME_MAP\"", "}", "elem {", "  name: \"Ethernet0:2\"", "}", "]", "The GetResponse is below", "-------------------------", "", "oid:0x1500000000067a"], "stderr_lines": [], "_ansible_no_log": false, "failed": false}
01/07/2025 19:00:49 base._run                                L0071 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell, args=["python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:3             -xt COUNTERS_DB -o \"xxxxxxxxx\"             "], kwargs={"module_ignore_errors": true}                                                                                                                                                                                                                                                                                                                          01/07/2025 19:00:50 base._run                                L0108 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell Result => {"changed": true, "end": "2025-07-01 19:00:49.997748", "stdout": "Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path\n ------------------------- \n [elem {\n  name: \"COUNTERS_QUEUE_NAME_MAP\"\n}\nelem {\n  name: \"Ethernet0:3\"\n}\n]\nThe GetResponse is below\n-------------------------\n\noid:0x1500000000067b", "cmd": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:3             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "rc": 0, "start": "2025-07-01 19:00:49.798159", "stderr": "", "delta": "0:00:00.199589", "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:3             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "removes": null, "argv": null, "warn": true, "chdir": null, "stdin_add_newline": true, "stdin": null}}, "stdout_lines": ["Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path", " ------------------------- ", " [elem {", "  name: \"COUNTERS_QUEUE_NAME_MAP\"", "}", "elem {", "  name: \"Ethernet0:3\"", "}", "]", "The GetResponse is below", "-------------------------", "", "oid:0x1500000000067b"], "stderr_lines": [], "_ansible_no_log": false, "failed": false}
01/07/2025 19:00:50 base._run                                L0071 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell, args=["python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:4             -xt COUNTERS_DB -o \"xxxxxxxxx\"             "], kwargs={"module_ignore_errors": true}                                                                                                                                                                                                                                                                                                                          01/07/2025 19:00:50 base._run                                L0108 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell Result => {"changed": true, "end": "2025-07-01 19:00:50.381489", "stdout": "Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path\n ------------------------- \n [elem {\n  name: \"COUNTERS_QUEUE_NAME_MAP\"\n}\nelem {\n  name: \"Ethernet0:4\"\n}\n]\nThe GetResponse is below\n-------------------------\n\noid:0x1500000000067c", "cmd": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:4             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "rc": 0, "start": "2025-07-01 19:00:50.177327", "stderr": "", "delta": "0:00:00.204162", "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:4             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "removes": null, "argv": null, "warn": true, "chdir": null, "stdin_add_newline": true, "stdin": null}}, "stdout_lines": ["Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path", " ------------------------- ", " [elem {", "  name: \"COUNTERS_QUEUE_NAME_MAP\"", "}", "elem {", "  name: \"Ethernet0:4\"", "}", "]", "The GetResponse is below", "-------------------------", "", "oid:0x1500000000067c"], "stderr_lines": [], "_ansible_no_log": false, "failed": false}
01/07/2025 19:00:50 base._run                                L0071 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell, args=["python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:5             -xt COUNTERS_DB -o \"xxxxxxxxx\"             "], kwargs={"module_ignore_errors": true}                                                                                                                                                                                                                                                                                                                          01/07/2025 19:00:50 base._run                                L0108 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell Result => {"changed": true, "end": "2025-07-01 19:00:50.771962", "stdout": "Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path\n ------------------------- \n [elem {\n  name: \"COUNTERS_QUEUE_NAME_MAP\"\n}\nelem {\n  name: \"Ethernet0:5\"\n}\n]\nThe GetResponse is below\n-------------------------\n\noid:0x1500000000067d", "cmd": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:5             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "rc": 0, "start": "2025-07-01 19:00:50.566323", "stderr": "", "delta": "0:00:00.205639", "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:5             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "removes": null, "argv": null, "warn": true, "chdir": null, "stdin_add_newline": true, "stdin": null}}, "stdout_lines": ["Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path", " ------------------------- ", " [elem {", "  name: \"COUNTERS_QUEUE_NAME_MAP\"", "}", "elem {", "  name: \"Ethernet0:5\"", "}", "]", "The GetResponse is below", "-------------------------", "", "oid:0x1500000000067d"], "stderr_lines": [], "_ansible_no_log": false, "failed": false}
01/07/2025 19:00:50 base._run                                L0071 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell, args=["python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:6             -xt COUNTERS_DB -o \"xxxxxxxxx\"             "], kwargs={"module_ignore_errors": true}                                                                                                                                                                                                                                                                                                                          01/07/2025 19:00:51 base._run                                L0108 DEBUG  | /var/src/sonic-mgmt-int/tests/telemetry/test_telemetry.py::get_buffer_queues_cnt#47: [xxxxx-x] AnsibleModule::shell Result => {"changed": true, "end": "2025-07-01 19:00:51.236571", "stdout": "Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path\n ------------------------- \n [elem {\n  name: \"COUNTERS_QUEUE_NAME_MAP\"\n}\nelem {\n  name: \"Ethernet0:6\"\n}\n]\nThe GetResponse is below\n-------------------------\n\noid:0x1500000000067e", "cmd": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:6             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "rc": 0, "start": "2025-07-01 19:00:50.951268", "stderr": "", "delta": "0:00:00.285303", "invocation": {"module_args": {"creates": null, "executable": null, "_uses_shell": true, "strip_empty_ends": true, "_raw_params": "python /root/gnxi/gnmi_cli_py/py_gnmicli.py -g -t xx.xx.xxx.xxx             -p 50051 -m get -x COUNTERS_QUEUE_NAME_MAP/Ethernet0:6             -xt COUNTERS_DB -o \"xxxxxxxxx\"             ", "removes": null, "argv": null, "warn": true, "choir": null, "stdin_add_newline": true, "stdin": null}}, "stdout_lines": ["Performing GetRequest, encoding=JSON_IETF to xx.xx.xxx.xxx  with the following gNMI Path", " ------------------------- ", " [elem {", "  name: \"COUNTERS_QUEUE_NAME_MAP\"", "}", "elem {", "  name: \"Ethernet0:6\"", "}", "]", "The GetResponse is below", "-------------------------", "", "oid:0x1500000000067e"], "stderr_lines": [], "_ansible_no_log": false, "failed": false}

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[developfast]

as comments

[developfast]

is the yang_validate to False intentional?

[zbud-msft]

Yes, after modifying BUFFER_QUEUE and doing config reload, we will skip yang validation.

[developfast]

The variable name single_key is misleading. It actually indicates whether we have a single queue (not a range). Consider renaming to is_single_queue

[zbud-msft]

Agreed and changed.

[developfast]

Add a check to ensure interface_buffer_queues is not empty before accessing interface_buffer_queues[0]

[zbud-msft]

Done

[developfast]

nit: the logic here is pretty useful for other use cases too. Wanna split the logic out into a helper func to make it more readable?

[zbud-msft]

Can take this effort in some future PR.

[developfast]

can you make the failure message more specific?

[zbud-msft]

Done

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull Request Overview

This PR ensures the buffer‐queue count test correctly handles configurations where all queues for an interface are defined in a single range (e.g., Ethernet0|0-9) by splitting that range before removal, and adds a parameter to disable YANG validation on reload.

• Pass yang_validate=False into config_reload to skip YANG validation during test config loads.
• Before removing a buffer‐queue entry, detect and split grouped queue ranges into a single‐queue entry and a remaining range.
• Wrap the removal and assertions in a try/finally to restore the original configuration regardless of test outcome.

Comments suppressed due to low confidence (1)

tests/telemetry/test_telemetry.py:34

• [nitpick] Add a brief inline comment explaining why yang_validate=False is required here (e.g., to bypass YANG schema checks for test changes).

    config_reload(duthost, config_source='config_db', safe_reload=True, check_intf_up_ports=True,

[Copilot]

[nitpick] This triple-quoted string is a hanging literal used as a comment. Replace it with standard # comments to avoid leaving unused string literals in the function body.

Suggested change

	"""If all queues for that pool are in the same pool ex Ethernet0|0-9
	We will modify to separate the first queue and the remaining such
	that we get a separate entry for Ethernet0|0 and Ethernet0|1-9"""
	# If all queues for that pool are in the same pool, e.g., Ethernet0|0-9,
	# we will modify to separate the first queue and the remaining such
	# that we get a separate entry for Ethernet0|0 and Ethernet0|1-9.

[developfast]

lol so useless

[make1980]

[developfast]

lgtm

[zbud-msft]

Hi @yejianquan can you please help with merge/backport to 202505?

[StormLiangMS]

LGTM

[mssonicbld]

Cherry-pick PR to 202411: #19466

[mssonicbld]

Cherry-pick PR to 202505: #19501

[liuh-80]

@zbud-msft , the cherry-pick PR #19466 break test_telemetry.py on 202411 branch:

File "/var/src/sonic-mgmt/tests/telemetry/test_telemetry.py", line 34, in load_new_cfg
config_reload(duthost, config_source='config_db', safe_reload=True, check_intf_up_ports=True,
File "/var/src/sonic-mgmt/tests/common/plugins/loganalyzer/utils.py", line 24, in decorated
res = func(*args, **kwargs)
TypeError: config_reload() got an unexpected keyword argument 'yang_validate'

Maybe other branch also broken.