[Counters] Improve performance by polling only configured ports buffer queue/pg counters

[shlomibitton]

What I did

Currently in SONiC all ports queue and pg counters are created by default with the max possible amount of counters.
This feature change this behavior to poll only configured counters provided by the config DB BUFFER_PG and BUFFER_QUEUE tables.
If no tables are present in the DB, no counters will be created for ports.
Filter the unwanted queues/pgs returned by SAI API calls and skip the creation of these queue/pg counters.
Also allow creating/removing counters on runtime if buffer PG/Queue is configured or removed.

Why I did it

Improve performance by filtering unconfigured queue/pg counters on init.

How I verified it

Check after enabling the counters, if configured counters created in Counters DB according to the configurations.
Add/Remove buffer PG/Queue configurations and observe the corresponding counters created/removed accordingly.
New UT added to verify this flow.

Details if related

[azure-pipelines]

Supported commands

• help:
• Get descriptions, examples and documentation about supported commands
• Example: help "command_name"
• list:
• List all pipelines for this repository using a comment.
• Example: "list"
• run:
• Run all pipelines or specific pipelines for this repository using a comment. Use this command by itself to trigger all related pipelines, or specify specific pipelines to run.
• Example: "run" or "run pipeline_name, pipeline_name, pipeline_name"
• where:
• Report back the Azure DevOps orgs that are related to this repository and org
• Example: "where"

See additional documentation.

[azure-pipelines]

Commenter does not have sufficient privileges for PR 2143 in repo Azure/sonic-swss

[azure-pipelines]

Commenter does not have sufficient privileges for PR 2143 in repo Azure/sonic-swss

[mssonicbld]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/AzurePipelines run LGTM analysis: C/C++

[azure-pipelines]

No pipelines are associated with this pull request.

[mssonicbld]

/AzurePipelines run LGTM

[azure-pipelines]

No pipelines are associated with this pull request.

[mssonicbld]

/AzurePipelines run Azure.sonic-swss (Test vstest)

[azure-pipelines]

No pipelines are associated with this pull request.

[shlomibitton]

/azpw run Azure.sonic-swss

[mssonicbld]

/AzurePipelines run Azure.sonic-swss

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[liat-grozovik]

@stepanblyschak could you please review this?

[liat-grozovik]

/azp run Azure.sonic-swss

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[shlomibitton]

/azpw run Azure.sonic-swss

[mssonicbld]

/AzurePipelines run Azure.sonic-swss

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[yxieca]

@shlomibitton I tried again, still having PR test failures with:

        "stdin": null, 
        "stdin_add_newline": true, 
        "strip_empty_ends": true, 
        "warn": true
    }
}, 
"msg": "non-zero return code", 
"rc": 1, 
"start": "2022-06-23 10:17:24.697107", 
"stderr": "Traceback (most recent call last):\n  File \"/usr/local/bin/watermarkstat\", line 315, in <module>\n    main()\n  File \"/usr/local/bin/watermarkstat\", line 310, in main\n    watermarkstat.print_all_stat(table_prefix, args.type)\n  File \"/usr/local/bin/watermarkstat\", line 261, in print_all_stat\n    data = self.get_counters(table_prefix,\n  File \"/usr/local/bin/watermarkstat\", line 237, in get_counters\n    elif fields[pos] != STATUS_NA:\nIndexError: list index out of range", 
"stderr_lines": [
    "Traceback (most recent call last):", 
    "  File \"/usr/local/bin/watermarkstat\", line 315, in <module>", 
    "    main()", 
    "  File \"/usr/local/bin/watermarkstat\", line 310, in main", 
    "    watermarkstat.print_all_stat(table_prefix, args.type)", 
    "  File \"/usr/local/bin/watermarkstat\", line 261, in print_all_stat", 
    "    data = self.get_counters(table_prefix,", 
    "  File \"/usr/local/bin/watermarkstat\", line 237, in get_counters", 
    "    elif fields[pos] != STATUS_NA:", 
    "IndexError: list index out of range"
], 
"stdout": "", 
"stdout_lines": []

}

[shlomibitton]

@yxieca did you try it with the fix I provided for sonic-utilities?
sonic-net/sonic-utilities#2220

[yxieca]

@yxieca did you try it with the fix I provided for sonic-utilities? Azure/sonic-utilities#2220

Ah, I see, I jumped the gun. The utility PR is still open.