Skip to content

MIGSMSFT-767 test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0]#15801

Merged
yejianquan merged 2 commits intosonic-net:masterfrom
zhixzhu:pfcwd_basic
Dec 3, 2024
Merged

MIGSMSFT-767 test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0]#15801
yejianquan merged 2 commits intosonic-net:masterfrom
zhixzhu:pfcwd_basic

Conversation

@zhixzhu
Copy link
Contributor

@zhixzhu zhixzhu commented Nov 28, 2024

Description of PR

Summary:
Fixes # (issue)
https://migsonic.atlassian.net/browse/MIGSMSFT-767 [T2 IXIA] failures in test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0]

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 202012
  • 202205
  • 202305
  • 202311
  • 202405

Approach

What is the motivation for this PR?

Fix the failures of test_multidut_pfcwd_basic_with_snappi.py

How did you do it?

Data flow 2 dropped packets.
2 flows in default-voq used single voq, which caused tail drop before pfc pause was triggered. Decrease flow number from 2 to 1. Since backplane port's bandwidth is 200G, also decrease the traffic rate from 99.98% to 49.99%

How did you verify/test it?

Verified on T2 ixia testbed.

----------------------- generated xml file: /run_logs/ixia/18470/2024-11-28-06-00-02/tr_2024-11-28-06-00-02.xml -----------------------
INFO:root:Can not get Allure report URL. Please check logs
------------------------------------------------------- live log sessionfinish --------------------------------------------------------
08:42:08 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
======================================================= short test summary info =======================================================
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio[multidut_port_info0-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio[multidut_port_info0-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio[multidut_port_info0-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio[multidut_port_info0-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_reboot[multidut_port_info0-cold-yy39top-lc4|3-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_reboot[multidut_port_info0-cold-yy39top-lc4|3-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_reboot[multidut_port_info0-cold-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_reboot[multidut_port_info0-cold-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_service_restart[multidut_port_info0-True-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_service_restart[multidut_port_info0-False-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_restart_service[multidut_port_info0-True-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_restart_service[multidut_port_info0-False-swss]
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:142: Reboot type warm is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:142: Reboot type fast is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:190: Reboot type warm is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:190: Reboot type fast is not supported on cisco-8000 switches
======================================= 12 passed, 8 skipped, 15 warnings in 9723.32s (2:42:03) =======================================
sonic@snappi-sonic-mgmt-vanilla-202405-t2:/data/tests$ 

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

@zhixzhu zhixzhu marked this pull request as ready for review November 28, 2024 22:45

flow1_delay_sec = restore_time_sec / 2 + WARM_UP_TRAFFIC_DUR
flow1_dur_sec = pfc_storm_dur_sec
flow1_delay_sec = detect_time_sec + poll_interval_sec
Copy link
Contributor

@sdszhang sdszhang Dec 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Warm up delay need to be added as pause traffic starts after warm_up traffic.
  • can you elaborate more on why the previous delay and duration was incorrect?
    The existing testing:
    PFC pause sending time: [1, 2]
    data flow1 sending time: [1.2, 2.2]
    with this setting, the traffic should be dropped once pfcwd is detected around 1.4 second?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data flow1 sending time: [1.4, 2.4]
pfc storm will be detected at ~1.8 second, and restored at ~2.8 second.
data flow1 will be blocked when pfc pause is triggered and dropped after pfc strom is detected. The packet drop of data flow1 will be 99.9%.
The original delay and duration should be good. Removed this change.

@zhixzhu zhixzhu changed the title MIGSMSFT-767 Fix issue on test flow delay time and duration time MIGSMSFT-767 [T2 IXIA] failures in test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0] Dec 3, 2024
@zhixzhu zhixzhu changed the title MIGSMSFT-767 [T2 IXIA] failures in test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0] MIGSMSFT-767 test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0] Dec 3, 2024
@yejianquan yejianquan merged commit 18b4f2f into sonic-net:master Dec 3, 2024
mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Dec 3, 2024
…of Data Flow 2 (0.1140553129761207) should be in [0, 0] (sonic-net#15801)

Description of PR
Summary:
Fixes # (issue)
https://migsonic.atlassian.net/browse/MIGSMSFT-767 [T2 IXIA] failures in test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0]

Approach
What is the motivation for this PR?
Fix the failures of test_multidut_pfcwd_basic_with_snappi.py

How did you do it?
Data flow 2 dropped packets.
2 flows in default-voq used single voq, which caused tail drop before pfc pause was triggered. Decrease flow number from 2 to 1. Since backplane port's bandwidth is 200G, also decrease the traffic rate from 99.98% to 49.99%

How did you verify/test it?
Verified on T2 ixia testbed.

----------------------- generated xml file: /run_logs/ixia/18470/2024-11-28-06-00-02/tr_2024-11-28-06-00-02.xml -----------------------
INFO:root:Can not get Allure report URL. Please check logs
------------------------------------------------------- live log sessionfinish --------------------------------------------------------
08:42:08 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
======================================================= short test summary info =======================================================
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio[multidut_port_info0-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio[multidut_port_info0-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio[multidut_port_info0-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio[multidut_port_info0-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_reboot[multidut_port_info0-cold-yy39top-lc4|3-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_reboot[multidut_port_info0-cold-yy39top-lc4|3-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_reboot[multidut_port_info0-cold-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_reboot[multidut_port_info0-cold-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_service_restart[multidut_port_info0-True-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_service_restart[multidut_port_info0-False-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_restart_service[multidut_port_info0-True-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_restart_service[multidut_port_info0-False-swss]
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:142: Reboot type warm is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:142: Reboot type fast is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:190: Reboot type warm is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:190: Reboot type fast is not supported on cisco-8000 switches
======================================= 12 passed, 8 skipped, 15 warnings in 9723.32s (2:42:03) =======================================
sonic@snappi-sonic-mgmt-vanilla-202405-t2:/data/tests$ 

Signed-off-by: Zhixin Zhu <[email protected]>
@mssonicbld
Copy link
Collaborator

Cherry-pick PR to 202405: #15860

mssonicbld pushed a commit that referenced this pull request Dec 5, 2024
…of Data Flow 2 (0.1140553129761207) should be in [0, 0] (#15801)

Description of PR
Summary:
Fixes # (issue)
https://migsonic.atlassian.net/browse/MIGSMSFT-767 [T2 IXIA] failures in test_pfcwd_basic_single_lossless_prio_reboot: Loss rate of Data Flow 2 (0.1140553129761207) should be in [0, 0]

Approach
What is the motivation for this PR?
Fix the failures of test_multidut_pfcwd_basic_with_snappi.py

How did you do it?
Data flow 2 dropped packets.
2 flows in default-voq used single voq, which caused tail drop before pfc pause was triggered. Decrease flow number from 2 to 1. Since backplane port's bandwidth is 200G, also decrease the traffic rate from 99.98% to 49.99%

How did you verify/test it?
Verified on T2 ixia testbed.

----------------------- generated xml file: /run_logs/ixia/18470/2024-11-28-06-00-02/tr_2024-11-28-06-00-02.xml -----------------------
INFO:root:Can not get Allure report URL. Please check logs
------------------------------------------------------- live log sessionfinish --------------------------------------------------------
08:42:08 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
======================================================= short test summary info =======================================================
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio[multidut_port_info0-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio[multidut_port_info0-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio[multidut_port_info0-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio[multidut_port_info0-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_reboot[multidut_port_info0-cold-yy39top-lc4|3-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_reboot[multidut_port_info0-cold-yy39top-lc4|3-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_reboot[multidut_port_info0-cold-True]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_reboot[multidut_port_info0-cold-False]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_service_restart[multidut_port_info0-True-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_single_lossless_prio_service_restart[multidut_port_info0-False-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_restart_service[multidut_port_info0-True-swss]
PASSED snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py::test_pfcwd_basic_multi_lossless_prio_restart_service[multidut_port_info0-False-swss]
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:142: Reboot type warm is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:142: Reboot type fast is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:190: Reboot type warm is not supported on cisco-8000 switches
SKIPPED [2] snappi_tests/multidut/pfcwd/test_multidut_pfcwd_basic_with_snappi.py:190: Reboot type fast is not supported on cisco-8000 switches
======================================= 12 passed, 8 skipped, 15 warnings in 9723.32s (2:42:03) =======================================
sonic@snappi-sonic-mgmt-vanilla-202405-t2:/data/tests$ 

Signed-off-by: Zhixin Zhu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants