Improvising qos tests by tunning the qos params for single_asic, single_dut_multi_asic and multi_dut by ansrajpu-git · Pull Request #8222 · sonic-net/sonic-mgmt

ansrajpu-git · 2023-05-05T16:13:05Z

Description of PR

This PR is in continuation of PR# #8149
which was originally part of PR# #6946

The existing QoS (test_qos_sai.py) is written to accommodate a single asic on a single Dut. But, we require the same tests to be executed against a T2 chassis (with single/multi-asic linecards) and multi-asic pizza boxes.

Summary:
This PR is in continuation to above PR's mentioned

Fixes # (issue)

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

201911
202012
202205

Approach

What is the motivation for this PR?

1.Qos test cases failed with intermittent errors

How did you do it?

Two issues are addressed here :

1.The dscp queue mapping for LossyQueue Test changed in config file to map to queue 1 of traffic-class instead of 0 since disabling the tx and filling up the queue 0 prevents the lacp packets going out and port channel goes down

2.During Qos test on transmission disable and enable, sometimes on test failure the port dangles in a transmission disable state and did not recover. Switching the step to enable the transmission port before the BCMSAI credit-watchdog enable , eradicate the test failure due to bad port state.

How did you verify/test it?

Executed qos testcases on for single_asic ,single_dut_multi_asic & multi_dut

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

mssonicbld · 2023-05-05T16:14:06Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Passed check for added large files..............................................Passed check python ast.........................................................Failed - hook id: check-ast - exit code: 1 tests/saitests/py3/sai_base_test.py: failed parsing with CPython 3.8.10: Traceback (most recent call last): File "/home/AzDevOps/.cache/pre-commit/repoojeenau5/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_ast.py", line 21, in main ast.parse(f.read(), filename=filename) File "/usr/lib/python3.8/ast.py", line 47, in parse return compile(source, filename, mode, flags, File "tests/saitests/py3/sai_base_test.py", line 12 <<<<<<< HEAD ^ SyntaxError: invalid syntax ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

mssonicbld · 2023-05-05T21:38:31Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Passed check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_base_test.py:12:1: F401 'json' imported but unused tests/saitests/py3/sai_base_test.py:24:1: F811 redefinition of unused 'socket' from line 13 tests/saitests/py3/sai_base_test.py:138:5: E303 too many blank lines (3) tests/saitests/py3/sai_qos_tests.py:3115:12: E114 indentation is not a multiple of 4 (comment) check conditional mark sort..............................................Passed

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

xwjiang-ms · 2023-05-08T08:25:25Z

pre-commit check is a mandatory check, please fix detected issues

mssonicbld · 2023-05-15T22:51:14Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Passed check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_base_test.py:12:1: F401 'json' imported but unused tests/saitests/py3/sai_base_test.py:24:1: F811 redefinition of unused 'socket' from line 13 tests/saitests/py3/sai_base_test.py:138:5: E303 too many blank lines (3) tests/saitests/py3/sai_qos_tests.py:442:17: E265 block comment should start with '# ' tests/saitests/py3/sai_qos_tests.py:3121:12: E114 indentation is not a multiple of 4 (comment) check conditional mark sort..............................................Passed

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

mssonicbld · 2023-06-01T17:25:35Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Passed check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_qos_tests.py:451:17: E265 block comment should start with '# ' check conditional mark sort..............................................Passed

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

vmittal-msft · 2023-07-10T17:50:04Z

This PR is dependent on #8149. Once we have #8149 pushed in, we can push in this as well.

judyjoseph · 2023-07-25T21:41:56Z

@vmittal-msft Could you check and review this PR ?

vmittal-msft · 2023-08-01T19:04:16Z

tests/qos/files/qos_params.j2c.yaml

Lets remove it. This will be fixed by BRCM CSP.

tests/qos/files/qos_params.j2c.yaml

tests/saitests/py3/sai_base_test.py

tests/saitests/py3/sai_qos_tests.py

vmittal-msft · 2023-08-09T00:44:58Z

tests/qos/files/qos_params.j2c.yaml

please keep existing values as we will have fix for this.

mssonicbld · 2023-08-09T14:45:36Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Passed check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_base_test.py:183:21: F821 undefined name 'time' tests/saitests/py3/sai_base_test.py:190:93: E231 missing whitespace after ',' tests/saitests/py3/sai_base_test.py:193:5: E303 too many blank lines (2) tests/saitests/py3/sai_base_test.py:206:21: F821 undefined name 'time' tests/saitests/py3/sai_qos_tests.py:3094:12: E114 indentation is not a multiple of 4 (comment) flake8...............................................(no files to check)Skipped check conditional mark sort..............................................Passed

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

ansrajpu-git · 2023-08-22T17:32:39Z

@XuChen-MSFT ,@wsycqyz ,kindly review & approve

mssonicbld · 2023-08-28T09:21:23Z

@ansrajpu-git PR conflicts with 202205 branch

mssonicbld · 2023-08-28T09:21:24Z

Cherry-pick PR to 202305: #9703

…le_dut_multi_asic and multi_dut (sonic-net#8222) * QoS tunning for LossyQueueTest * Making assert specific to dnx for DscpQueueMapping qos_test fix flake8 errors flake8 issue resolved * Recommiting the changes of qos.yml to qos_params_j2c.yml * Reverting dscp to 7 as this is fixed in BRCM CSP * Revert "Making assert specific to dnx for DscpQueueMapping qos_test" This reverts commit dcbf90fa1bbdb3778d9ef86a202c30d86bd08114. * changing dscp value back to 8 * tx_enable/disable retry check corrected * Flake8 errore fixed * Making check to run only for chassis in LossyQueue * Adding check for sai_base for tx_enable/disable

…le_dut_multi_asic and multi_dut (#8222) (#9703) This PR is in continuation of PR# #8149 which was originally part of PR# #6946 The existing QoS (test_qos_sai.py) is written to accommodate a single asic on a single Dut. But, we require the same tests to be executed against a T2 chassis (with single/multi-asic linecards) and multi-asic pizza boxes. What is the motivation for this PR? 1.Qos test cases failed with intermittent errors How did you do it? Two issues are addressed here : 1.The dscp queue mapping for LossyQueue Test changed in config file to map to queue 1 of traffic-class instead of 0 since disabling the tx and filling up the queue 0 prevents the lacp packets going out and port channel goes down 2.During Qos test on transmission disable and enable, sometimes on test failure the port dangles in a transmission disable state and did not recover. Switching the step to enable the transmission port before the BCMSAI credit-watchdog enable , eradicate the test failure due to bad port state. How did you verify/test it? Executed qos testcases on for single_asic ,single_dut_multi_asic & multi_dut Co-authored-by: ansrajpu-git <[email protected]>

… single_asic, single_dut_multi_asic and multi_dut (sonic-net#8222) from master to 202205 * QoS tunning for LossyQueueTest * Making assert specific to dnx for DscpQueueMapping qos_test fix flake8 errors flake8 issue resolved * Recommiting the changes of qos.yml to qos_params_j2c.yml * Reverting dscp to 7 as this is fixed in BRCM CSP * Revert "Making assert specific to dnx for DscpQueueMapping qos_test" This reverts commit dcbf90fa1bbdb3778d9ef86a202c30d86bd08114. * changing dscp value back to 8 * tx_enable/disable retry check corrected * Flake8 errore fixed * Making check to run only for chassis in LossyQueue * Adding check for sai_base for tx_enable/disable

… single_asic, single_dut_multi_asic and multi_dut (#8222) from master to 202205 (#10209) Master PR #8222 QoS tunning for LossyQueueTest Recommiting the changes of qos.yml to qos_params_j2c.yml tx_enable/disable retry check corrected Making check to run only for chassis in LossyQueue Adding check for sai_base for tx_enable/disable

…le_dut_multi_asic and multi_dut (sonic-net#8222) * QoS tunning for LossyQueueTest * Making assert specific to dnx for DscpQueueMapping qos_test fix flake8 errors flake8 issue resolved * Recommiting the changes of qos.yml to qos_params_j2c.yml * Reverting dscp to 7 as this is fixed in BRCM CSP * Revert "Making assert specific to dnx for DscpQueueMapping qos_test" This reverts commit dcbf90fa1bbdb3778d9ef86a202c30d86bd08114. * changing dscp value back to 8 * tx_enable/disable retry check corrected * Flake8 errore fixed * Making check to run only for chassis in LossyQueue * Adding check for sai_base for tx_enable/disable

wenyiz2021 requested a review from vmittal-msft May 5, 2023 21:39

rlhui assigned ansrajpu-git May 5, 2023

ansrajpu-git changed the title ~~Enhance qos tests to support single-asic, multi-asic, and multi-dut testing~~ Improvising qos tests by tunning the qos params for single_asic, single_dut_multi_asic and multi_dut May 8, 2023

ansrajpu-git requested review from neethajohn, prgeor, wangxin and yxieca as code owners May 15, 2023 22:32

ansrajpu-git force-pushed the qos_fix branch from 8506e0d to ae9daa4 Compare May 15, 2023 22:50

ansrajpu-git force-pushed the qos_fix branch from ae9daa4 to 9f9d3fa Compare June 1, 2023 17:24

ansrajpu-git requested review from XuChen-MSFT and wsycqyz as code owners June 1, 2023 17:24

ansrajpu-git force-pushed the qos_fix branch from ae191e6 to f1a52b5 Compare July 19, 2023 14:51

ansrajpu-git force-pushed the qos_fix branch 2 times, most recently from 415469e to 3385718 Compare July 28, 2023 15:27

vmittal-msft added Request for 202205 branch chassis labels Jul 31, 2023

vmittal-msft reviewed Aug 1, 2023

View reviewed changes

tests/qos/files/qos_params.j2c.yaml Outdated

Copy link

Contributor

vmittal-msft Aug 1, 2023 •

edited

Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets remove it. This will be fixed by BRCM CSP.

vmittal-msft reviewed Aug 1, 2023

View reviewed changes

tests/qos/files/qos_params.j2c.yaml Outdated Show resolved Hide resolved

vmittal-msft reviewed Aug 1, 2023

View reviewed changes

tests/saitests/py3/sai_base_test.py Outdated Show resolved Hide resolved

vmittal-msft reviewed Aug 1, 2023

View reviewed changes

tests/saitests/py3/sai_qos_tests.py Outdated Show resolved Hide resolved

ansrajpu-git force-pushed the qos_fix branch 2 times, most recently from c0dc5c8 to a67cc10 Compare August 8, 2023 14:46

vmittal-msft reviewed Aug 9, 2023

View reviewed changes

tests/qos/files/qos_params.j2c.yaml Outdated

Copy link

Contributor

vmittal-msft Aug 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please keep existing values as we will have fix for this.

ansrajpu-git added 2 commits August 9, 2023 10:40

changing dscp value back to 8

0a351f1

tx_enable/disable retry check corrected

3a033eb

ansrajpu-git force-pushed the qos_fix branch from 9b31663 to 3a033eb Compare August 9, 2023 14:44

ansrajpu-git added 2 commits August 9, 2023 12:01

Flake8 errore fixed

b3e4627

Making check to run only for chassis in LossyQueue

608d15a

vmittal-msft previously approved these changes Aug 11, 2023

View reviewed changes

Adding check for sai_base for tx_enable/disable

45dccda

ansrajpu-git dismissed vmittal-msft’s stale review via 45dccda August 15, 2023 00:57

vmittal-msft approved these changes Aug 21, 2023

View reviewed changes

vmittal-msft merged commit 7133697 into sonic-net:master Aug 22, 2023

wangxin added Request for 202305 branch Approved for 202205 branch Approved for 202305 branch labels Aug 28, 2023

mssonicbld added the Cherry Pick Conflict_202205 label Aug 28, 2023

mssonicbld added the Created PR to 202305 branch label Aug 28, 2023

mssonicbld mentioned this pull request Aug 28, 2023

[action] [PR:8222] Improvising qos tests by tunning the qos params for single_asic, single_dut_multi_asic and multi_dut #9703

Merged

6 tasks

mssonicbld added Included in 202305 branch and removed Created PR to 202305 branch labels Aug 29, 2023

ansrajpu-git mentioned this pull request Oct 2, 2023

[Cherry-pick] Improvising qos tests by tunning the qos params for…#8222 #10209

Merged

6 tasks

wangxin added the Included in 202205 branch label Oct 23, 2023

Conversation

ansrajpu-git commented May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

mssonicbld commented May 5, 2023

Uh oh!

mssonicbld commented May 5, 2023

Uh oh!

xwjiang-ms commented May 8, 2023

Uh oh!

mssonicbld commented May 15, 2023

Uh oh!

mssonicbld commented Jun 1, 2023

Uh oh!

vmittal-msft commented Jul 10, 2023

Uh oh!

judyjoseph commented Jul 25, 2023

Uh oh!

vmittal-msft Aug 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vmittal-msft Aug 9, 2023

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Aug 9, 2023

Uh oh!

ansrajpu-git commented Aug 22, 2023

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

ansrajpu-git commented May 5, 2023 •

edited

Loading

vmittal-msft Aug 1, 2023 •

edited

Loading