[Snappi] PFCWD Tolerance Fixes by developfast · Pull Request #9567 · sonic-net/sonic-mgmt

developfast · 2023-08-21T04:16:51Z

Description of PR

Summary: PFCWD tests deviations for certain basic and multi node tests did not exist since there was no source of empirical data available for devices until recently. These tests require empirical tuning for the specific device the test runs on. This PR introduces certain allowable deviations based on device type for basic and multi node PFCWD tests.

Fixes # (issue)

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

201911
202012
202205

Approach

What is the motivation for this PR?

Fix PFCWD snappi tests running on nightly

How did you do it?

Introduce new acceptable tolerances

How did you verify/test it?

Ran on lab device

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

tests/snappi_tests/pfcwd/files/pfcwd_multi_node_helper.py

mssonicbld · 2023-08-28T08:39:21Z

@developfast PR conflicts with 202012 branch

mssonicbld · 2023-08-28T08:39:23Z

@developfast PR conflicts with 202205 branch

mssonicbld · 2023-08-28T08:42:18Z

Cherry-pick PR to 202305: #9697

Signed-off-by: Austin Pham <[email protected]>

fix: migrate sonic-net#7370 to multidut code Signed-off-by: Austin Pham <[email protected]> fix: migrate sonic-net#9567 to multidut code Signed-off-by: Austin Pham <[email protected]> fix: migrate sonic-net#7367 to multidut code Signed-off-by: Austin Pham <[email protected]> fix: migrate sonic-net#7712 Signed-off-by: Austin Pham <[email protected]> fix: migrate sonic-net#7712 to pfcwd_multidut_runtime_traffic_helper Signed-off-by: Austin Pham <[email protected]> chore: update a2a parameterize Signed-off-by: Austin Pham <[email protected]>

Summary: Update the missing T0 T1 tests feature in multidut Fixes # (issue) 30929279 The following PR were migrated for pfcwd [Snappi] Increase number of calls for ARP wait #7370 [Snappi] PFCWD Tolerance Fixes #9567 [Snappi] Modifications to pfcwd a2a test design for mlnx 4600c #7367 to multidut code [Snappi] Modified pfcwd tests to add warm-up traffic #7712 Signed-off-by: Austin Pham <[email protected]>

Summary: Update the missing T0 T1 tests feature in multidut Fixes # (issue) 30929279 The following PR were migrated for pfcwd - sonic-net/sonic-mgmt#7370 - sonic-net/sonic-mgmt#9567 - sonic-net/sonic-mgmt#7367 to multidut code - sonic-net/sonic-mgmt#7712 ### Type of change  - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202012 - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 ### Approach #### What is the motivation for this PR? #### How did you do it? #### How did you verify/test it? #### Any platform specific information? #### Supported testbed topology if it's a new test case? ### Documentation  snappi_tests/multidut/pfcwd/test_multidut_pfcwd_m2o_with_snappi.py ``` -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: /var/src/sonic-mgmt_vmsvc5-t2-8800-ixia_669de75fbbfea63e8b73b319/tests/logs/snappi_tests/multidut/pfcwd/test_multidut_pfcwd_m2o_with_snappi.xml - ================== 4 passed, 7 warnings in 1235.72s (0:20:35) ================== ``` snappi_tests/multidut/pfcwd/test_multidut_pfcwd_burst_storm_with_snappi.py ``` -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: /var/src/sonic-mgmt_vmsvc5-t2-8800-ixia_669de75fbbfea63e8b73b319/tests/logs/snappi_tests/multidut/pfcwd/test_multidut_pfcwd_burst_storm_with_snappi.xml - ================== 2 passed, 5 warnings in 858.60s (0:14:18) =================== ``` snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py (known failure on T2, fail as expected) ``` SKIPPED [4] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:190: Reboot type fast is not supported on cisco-8000 switches FAILED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_restart_service[multidut_port_info0-True-swss] ===== 1 failed, 23 passed, 16 skipped, 27 warnings in 27779.49s (7:42:59) ====== ``` snappi_tests/multidut/pfcwd/test_multidut_pfcwd_a2a_with_snappi.py ``` -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: /var/src/sonic-mgmt_vmsvc5-t2-8800-ixia_669de75fbbfea63e8b73b319/tests/logs/snappi_tests/multidut/pfcwd/test_multidut_pfcwd_a2a_with_snappi.xml - ================== 4 passed, 7 warnings in 1157.40s (0:19:17) ================== ``` snappi_tests/multidut/pfcwd/test_multidut_pfcwd_runtime_traffic_with_snappi.py ``` -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: /var/src/sonic-mgmt_vmsvc5-t2-8800-ixia_669de75fbbfea63e8b73b319/tests/logs/snappi_tests/multidut/pfcwd/test_multidut_pfcwd_runtime_traffic_with_snappi.xml - ================== 2 passed, 6 warnings in 1162.35s (0:19:22) ================== ```

Summary: Update the missing T0 T1 tests feature in multidut Fixes # (issue) 30929279 The following PR were migrated for pfcwd [Snappi] Increase number of calls for ARP wait sonic-net#7370 [Snappi] PFCWD Tolerance Fixes sonic-net#9567 [Snappi] Modifications to pfcwd a2a test design for mlnx 4600c sonic-net#7367 to multidut code [Snappi] Modified pfcwd tests to add warm-up traffic sonic-net#7712 Signed-off-by: Austin Pham <[email protected]>

Summary: Update the missing T0 T1 tests feature in multidut Fixes # (issue) 30929279 The following PR were migrated for pfcwd [Snappi] Increase number of calls for ARP wait #7370 [Snappi] PFCWD Tolerance Fixes #9567 [Snappi] Modifications to pfcwd a2a test design for mlnx 4600c #7367 to multidut code [Snappi] Modified pfcwd tests to add warm-up traffic #7712 Signed-off-by: Austin Pham <[email protected]>

Summary: Update the missing T0 T1 tests feature in multidut Fixes # (issue) 30929279 The following PR were migrated for pfcwd [Snappi] Increase number of calls for ARP wait sonic-net#7370 [Snappi] PFCWD Tolerance Fixes sonic-net#9567 [Snappi] Modifications to pfcwd a2a test design for mlnx 4600c sonic-net#7367 to multidut code [Snappi] Modified pfcwd tests to add warm-up traffic sonic-net#7712 Signed-off-by: Austin Pham <[email protected]>

pfcwd tolerance fix

edca7d7

developfast requested review from baiwei0427 and neethajohn August 21, 2023 04:17

developfast added Request for 202012 branch Request for 202205 branch Request for 202305 branch labels Aug 21, 2023

baiwei0427 reviewed Aug 21, 2023

View reviewed changes

tests/snappi_tests/pfcwd/files/pfcwd_multi_node_helper.py Outdated Show resolved Hide resolved

baiwei0427 reviewed Aug 21, 2023

View reviewed changes

tests/snappi_tests/pfcwd/files/pfcwd_multi_node_helper.py Show resolved Hide resolved

developfast added 2 commits August 21, 2023 17:46

Update pfcwd_multi_node_helper.py

e18bb4c

Update pfcwd_multi_node_helper.py

15af0ef

baiwei0427 approved these changes Aug 22, 2023

View reviewed changes

neethajohn approved these changes Aug 22, 2023

View reviewed changes

developfast merged commit 783bd28 into sonic-net:master Aug 23, 2023

yxieca mentioned this pull request Aug 23, 2023

[test gap] Enhance testQosSaiDscpQueueMapping and testQosSaiDscpToPgMapping to cover IPinIP traffic #8789

Closed

wangxin added Approved for 202012 Branch Approved for 202205 branch Approved for 202305 branch labels Aug 28, 2023

mssonicbld added Cherry Pick Conflict_202012 Cherry Pick Conflict_202205 labels Aug 28, 2023

wangxin added Approved for 202305 branch and removed Approved for 202305 branch labels Aug 28, 2023

mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Aug 28, 2023

[Snappi] PFCWD Tolerance Fixes (sonic-net#9567)

4e6f83f

mssonicbld added the Created PR to 202305 branch label Aug 28, 2023

mssonicbld mentioned this pull request Aug 28, 2023

[action] [PR:9567] [Snappi] PFCWD Tolerance Fixes #9697

Merged

6 tasks

mssonicbld pushed a commit that referenced this pull request Aug 28, 2023

[Snappi] PFCWD Tolerance Fixes (#9567)

c4a5afe

mssonicbld added the Included in 202305 branch label Aug 28, 2023

mssonicbld removed the Created PR to 202305 branch label Aug 28, 2023

developfast added a commit to developfast/sonic-mgmt that referenced this pull request Aug 29, 2023

[Snappi] PFCWD Tolerance Fixes (sonic-net#9567)

ffd92a4

developfast mentioned this pull request Aug 29, 2023

[202012][Snappi] PFCWD Tolerance Fixes (#9567) #9749

Merged

6 tasks

developfast added a commit to developfast/sonic-mgmt that referenced this pull request Aug 29, 2023

[Snappi] PFCWD Tolerance Fixes (sonic-net#9567)

8d0397d

developfast mentioned this pull request Aug 29, 2023

[202205][Snappi] PFCWD Tolerance Fixes (#9567) #9750

Merged

6 tasks

yxieca pushed a commit that referenced this pull request Aug 30, 2023

[Snappi] PFCWD Tolerance Fixes (#9567) (#9750)

1317670

yxieca pushed a commit that referenced this pull request Aug 30, 2023

[Snappi] PFCWD Tolerance Fixes (#9567) (#9749)

a70f46c

wangxin added Included in 202012 branch Included in 202205 branch labels Sep 15, 2023

AharonMalkin pushed a commit to AharonMalkin/sonic-mgmt that referenced this pull request Jan 25, 2024

[Snappi] PFCWD Tolerance Fixes (sonic-net#9567)

b629689

sdszhang mentioned this pull request Jan 20, 2025

Test merge a2a with snappi #16587

Closed

11 tasks

auspham added a commit to auspham/sonic-mgmt that referenced this pull request Jan 21, 2025

fix: migrate sonic-net#9567 to multidut code

2e6c179

Signed-off-by: Austin Pham <[email protected]>

auspham added a commit to auspham/sonic-mgmt that referenced this pull request Jan 21, 2025

fix: migrate sonic-net#9567 to multidut code

2c40e05

Signed-off-by: Austin Pham <[email protected]>

auspham mentioned this pull request Jan 24, 2025

[T0 T1 merge] update multidut pfcwd tests #16657

Merged

11 tasks

mssonicbld mentioned this pull request Jan 29, 2025

[action] [PR:16657] [T0 T1 merge] update multidut pfcwd tests Azure/sonic-mgmt.msft#29

Merged

11 tasks

mssonicbld mentioned this pull request Jan 29, 2025

[action] [PR:16657] [T0 T1 merge] update multidut pfcwd tests #16707

Merged

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Snappi] PFCWD Tolerance Fixes#9567

[Snappi] PFCWD Tolerance Fixes#9567
developfast merged 3 commits intosonic-net:masterfrom
developfast:pfcwd_tolerance_fix

developfast commented Aug 21, 2023

Uh oh!

Uh oh!

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

developfast commented Aug 21, 2023

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

Uh oh!

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

mssonicbld commented Aug 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants