[Snappi] - Infra changes related to new testcases added for PFC, ECN. by amitpawar12 · Pull Request #13655 · sonic-net/sonic-mgmt

amitpawar12 · 2024-07-14T21:11:15Z

Description of PR

PR - 13215 details the list of the new test-cases that were to be added to test PFC and ECN.

This PR is for the purpose of adding the necessary infrastructure for the same. Additionally, a single test case to test line-rate (for both 100Gbps and 400Gbps) is added.

Summary:
Fixes # (issue)
#13654
#13653
#13215
#13752

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

Approach

What is the motivation for this PR?

Pull request is for specifically providing infrastructure to run the additional testcases.

How did you do it?

Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test.

Similarly, there is test_check which passes test-case uses for verification info. By default, lossless prio's 3 and 4 are used and lossy prio 0,1 and 2 are used.

Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps.

This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively.

A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. In case of single line-card multiple asic and multiple line-card scenario, if the ports cannot be fitted in existing line-cards, it will try and swap to see if they can be still fitted.

New gen_sys flows are used to generate test, background and pause flows. These are called upon by multidut_helper file. The creation of flows and number of flows (depending upon ingress and egress), and verification is done here.

The multi_dut calls run_sys test in traffic_generation file and expects test_stats file in return. The test_stats contains summary of test execution results from both IXIA and DUT.

The rest_py is used to generate the imix custom profile.

Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.

Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same.

At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name.

Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format.

There is important change in variables.py file. The line_card_choice is sent as type of list from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and speed.

Example - line_card_choice = ['chassis_slcsa_100Gbps','chassis_mlcma_100Gbps','chassis_slcma_400Gbps']
will specifically run the test for single 100Gbps line card single asic 100Gbps, multiple 100Gbps line card multiple asic and single 400Gbps line card of multiple asic.

Additional support added in infrastructure:

#13654
There is pytest module function to enable credit watchdog as the test finishes. This function does not pass any asic_value. However function definition expects asic_value to be passed. Addressed this issue by enabling packet_aging on all available asics.

#13653
The function to get lossless buffer packets is checking ingress lossless buffer profile for Cisco platform. Added Nokia specific code to return the ingress lossless buffer values for Nokia 7250 Broadcom DNX platform as well.

#13752
Added support to enable PFCWD in Forwarding mode.

How did you verify/test it?

The test was executed on local setup in local clone.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

sdszhang · 2024-07-18T01:50:00Z

@selldinesh @rraghav-cisco Pls review, Thanks.

sdszhang · 2024-07-24T05:59:32Z

@selldinesh @rraghav-cisco Could you pls review this PR before the weekly meeting? Thanks.

sdszhang · 2024-07-24T06:17:07Z

tests/common/snappi_tests/common_helpers.py

-    if total_pkts == "N/A":
-        total_pkts = "0"
+    if duthost.is_multi_asic:
+        if (int(port.split('Ethernet')[1])) in range(0, 144):


can we use duthost.get_port_asic_instance(port).get_asic_namespace() to avoid hard code?

sdszhang · 2024-07-24T06:17:25Z

tests/common/snappi_tests/common_helpers.py

+        None
+    """
+    if (duthost.is_multi_asic):
+        if (int(port.split('Ethernet')[1])) in range(0, 144):


same as above

sdszhang · 2024-07-24T06:41:59Z

tests/snappi_tests/multidut/systest/test_non_congestion_imix.py

+    # port_map is defined as port-speed combination.
+    # first two parameters are count of egress links and its speed.
+    # last two parameters are count of ingress links and its speed.
+    if '100Gbps' in line_card_choice:


can you use dictionary instead of array for port_map?

yxieca · 2024-07-24T15:29:18Z

@amitpawar12 this is a fairly large PR. I understand that you might need all changes to address all issues listed. Do you think you could split this PR up into multiple PRs, one deal at one issue at a time?

@selldinesh @sdszhang @rraghav-cisco for viz.

amitpawar12 · 2024-07-24T19:27:28Z

@amitpawar12 this is a fairly large PR. I understand that you might need all changes to address all issues listed. Do you think you could split this PR up into multiple PRs, one deal at one issue at a time?

@selldinesh @sdszhang @rraghav-cisco for viz.

We can discuss this in the community meeting and post the update here.

Thanks,
-A

sdszhang · 2024-07-24T23:07:03Z

@vmittal-msft @developfast for viz.

sdszhang · 2024-07-24T23:09:49Z

@amitpawar12 pls share an update on how to split this diff. Thanks.

developfast · 2024-07-24T23:10:55Z

this is a huge pr and will take a long time to review - can you pls split PRs by testcase?

amitpawar12 · 2024-07-25T21:40:00Z

@sdszhang , @developfast , @vmittal-msft , All,

We are planning to break the changes into following sections:

Issue resolution code (different PR for each issue).
Infra changes for new testcases (single PR for all infra changes in snappi_fixture.py, traffic_generation.py and common_helper.py).
Each individual script for new testcases (individual PR for each script).

Please let me know and I will start working on creating these PRs.

Once the PRs are in place, I will put an additional comment against each section above with PR details.

Thanks,
-A

developfast · 2024-07-25T22:58:46Z

Good with everything, on the second point I want to know how comprehensive is this infra change? Don't want it turning into another large PR

amitpawar12 · 2024-07-26T02:10:06Z

Good with everything, on the second point I want to know how comprehensive is this infra change? Don't want it turning into another large PR

@developfast
The infra PR has three major changes:

snappi_ports will be added based on 'speed' and 'count of ports' as defined in port_map.
the run_sys_traffic will run the test, additionally, it will capture DUT and IXIA logs to be written in CSV file at the end of the test. This will also capture summary info in text file.
the gen_sys_flows (test, background and pause flows) are now iterative.

For DUT statistics, I will raise an independent PR and will not be part of this infra-PR.

Please let me know.

amitpawar12 · 2024-07-29T15:47:45Z

@developfast , @sdszhang , @vmittal-msft

This PR has been broken down into following three PRs for easy review:

[Snappi] Support for DUT statistics - Interface counters, PFC counters, and Queue counters. #13848 :
This PR deals with support of various DUT statistics - like interface counters, PFC counters and Queue counters. Additional provision is there to clean-up these counters before test-case execution.
[Snappi] Infra changes for new PFC-ECN testcases. #13864 :
This PR is created to handle only infrastructure related changes. This includes modification in traffic_generation.py, snappi_fixtures.py etc.
[Snappi] New testcases for PFC-ECN. #13865 :
This is created for reviewing the testcases ONLY.

The PRs will require the merge in the above order only as item#3 is dependent on (1) and (2) and item#2 is dependent on item#1.

Thanks,
-A

veronica-arista · 2024-09-11T22:39:16Z

tests/common/snappi_tests/common_helpers.py


    buffer_pools = config_facts['BUFFER_POOL']
-    profile_name = 'ingress_lossless_pool' if is_cisco8000_platform else 'egress_lossless_pool'
+    profile_name = 'ingress_lossless_pool' if (is_cisco8000_platform or is_nokia_7250) else 'egress_lossless_pool'


Changes to use the egress_lossless_pool instead of the ingress_lossless_pool should apply to all Broadcom DNX based platforms.

Can this selection be made by checking for broadcom-dnx in the platform duthost facts?

Description of PR 67989d1312b1778681d6575b12b66aa42fdf05a7 Please review the commit-ID given above. Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process. This PR tracks are the infrastructure related changes required for the execution of the testcases. Note - PR #13848 needs to be merged in first before this PR is merged. Summary: Fixes # (issue) #13655 #13215 Type of change Bug fix Testbed and Framework(new/improvement) Test case(new/improvement) Back port request 202012 202205 202305 202311 202405 Approach What is the motivation for this PR? This PR tracks only the infrastructure related changes needed for addition of the new testcases. How did you do it? Important changes are listed below: Change directory - tests/common/snappi_tests/ Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs. Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used. Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used. The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return. A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test). Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed. At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key. Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval. From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test. How did you verify/test it? Test was executed on the local clone. Any platform specific information? These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms. co-authorized by: jianquanye@microsoft.com

Description of PR 67989d1312b1778681d6575b12b66aa42fdf05a7 Please review the commit-ID given above. Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process. This PR tracks are the infrastructure related changes required for the execution of the testcases. Note - PR sonic-net#13848 needs to be merged in first before this PR is merged. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Type of change Bug fix Testbed and Framework(new/improvement) Test case(new/improvement) Back port request 202012 202205 202305 202311 202405 Approach What is the motivation for this PR? This PR tracks only the infrastructure related changes needed for addition of the new testcases. How did you do it? Important changes are listed below: Change directory - tests/common/snappi_tests/ Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs. Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used. Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used. The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return. A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test). Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed. At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key. Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval. From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test. How did you verify/test it? Test was executed on the local clone. Any platform specific information? These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms. co-authorized by: jianquanye@microsoft.com

Description of PR This pull-request has changes specifically for the following commit-IDs: a82b489 180af4d 3da40bc This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added. Summary: Fixes # (issue) #13655 #13215 Approach What is the motivation for this PR? Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode). How did you do it? Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps. This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports. Testcases: a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py: -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN. b. tests/snappi_tests/pfc/test_pfc_port_congestion.py: -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities. c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py: -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress. How did you verify/test it? Test case was executed on local clone. Results of the verification: Test cases executed for 100Gbps interfaces. Two combinations - single-line-card-multi-asic and multiple-dut Non-congestion: 19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic 19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv PASSED [ 16%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic 19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv PASSED [ 33%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic 19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic 19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv PASSED [ 66%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic 19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv PASSED [ 83%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic 19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv PASSED [100%] Over-subscription: 20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic 20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv PASSED [ 12%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic 20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv PASSED [ 25%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic 20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv PASSED [ 37%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic 20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic 21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv PASSED [ 62%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic 21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv PASSED [ 75%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic 21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv PASSED [ 87%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic 21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv PASSED [100%] PFCWD: 01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv PASSED [ 10%] 01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv PASSED [ 20%] 01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv PASSED [ 30%] 01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv PASSED [ 40%] 01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv PASSED [ 50%] 02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv PASSED [ 60%] 02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv PASSED [ 70%] 02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv PASSED [ 80%] 02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv PASSED [ 90%] 02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv PASSED [100%] Any platform specific information? The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT. co-authorized by: jianquanye@microsoft.com

Description of PR This pull-request has changes specifically for the following commit-IDs: a82b489 180af4d 3da40bc This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Approach What is the motivation for this PR? Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode). How did you do it? Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps. This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports. Testcases: a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py: -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN. b. tests/snappi_tests/pfc/test_pfc_port_congestion.py: -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities. c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py: -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress. How did you verify/test it? Test case was executed on local clone. Results of the verification: Test cases executed for 100Gbps interfaces. Two combinations - single-line-card-multi-asic and multiple-dut Non-congestion: 19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic 19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv PASSED [ 16%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic 19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv PASSED [ 33%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic 19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic 19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv PASSED [ 66%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic 19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv PASSED [ 83%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic 19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv PASSED [100%] Over-subscription: 20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic 20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv PASSED [ 12%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic 20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv PASSED [ 25%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic 20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv PASSED [ 37%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic 20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic 21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv PASSED [ 62%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic 21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv PASSED [ 75%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic 21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv PASSED [ 87%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic 21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv PASSED [100%] PFCWD: 01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv PASSED [ 10%] 01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv PASSED [ 20%] 01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv PASSED [ 30%] 01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv PASSED [ 40%] 01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv PASSED [ 50%] 02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv PASSED [ 60%] 02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv PASSED [ 70%] 02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv PASSED [ 80%] 02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv PASSED [ 90%] 02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv PASSED [100%] Any platform specific information? The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT. co-authorized by: jianquanye@microsoft.com

Description of PR 67989d1312b1778681d6575b12b66aa42fdf05a7 Please review the commit-ID given above. Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process. This PR tracks are the infrastructure related changes required for the execution of the testcases. Note - PR #13848 needs to be merged in first before this PR is merged. Summary: Fixes # (issue) #13655 #13215 Type of change Bug fix Testbed and Framework(new/improvement) Test case(new/improvement) Back port request 202012 202205 202305 202311 202405 Approach What is the motivation for this PR? This PR tracks only the infrastructure related changes needed for addition of the new testcases. How did you do it? Important changes are listed below: Change directory - tests/common/snappi_tests/ Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs. Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used. Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used. The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return. A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test). Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed. At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key. Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval. From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test. How did you verify/test it? Test was executed on the local clone. Any platform specific information? These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms. co-authorized by: jianquanye@microsoft.com

Description of PR This pull-request has changes specifically for the following commit-IDs: a82b489 180af4d 3da40bc This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added. Summary: Fixes # (issue) #13655 #13215 Approach What is the motivation for this PR? Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode). How did you do it? Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps. This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports. Testcases: a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py: -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN. b. tests/snappi_tests/pfc/test_pfc_port_congestion.py: -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities. c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py: -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress. How did you verify/test it? Test case was executed on local clone. Results of the verification: Test cases executed for 100Gbps interfaces. Two combinations - single-line-card-multi-asic and multiple-dut Non-congestion: 19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic 19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv PASSED [ 16%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic 19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv PASSED [ 33%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic 19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic 19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv PASSED [ 66%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic 19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv PASSED [ 83%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic 19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv PASSED [100%] Over-subscription: 20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic 20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv PASSED [ 12%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic 20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv PASSED [ 25%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic 20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv PASSED [ 37%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic 20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic 21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv PASSED [ 62%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic 21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv PASSED [ 75%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic 21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv PASSED [ 87%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic 21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv PASSED [100%] PFCWD: 01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv PASSED [ 10%] 01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv PASSED [ 20%] 01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv PASSED [ 30%] 01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv PASSED [ 40%] 01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv PASSED [ 50%] 02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv PASSED [ 60%] 02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv PASSED [ 70%] 02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv PASSED [ 80%] 02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv PASSED [ 90%] 02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv PASSED [100%] Any platform specific information? The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT. co-authorized by: jianquanye@microsoft.com

Description of PR 67989d1312b1778681d6575b12b66aa42fdf05a7 Please review the commit-ID given above. Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process. This PR tracks are the infrastructure related changes required for the execution of the testcases. Note - PR sonic-net#13848 needs to be merged in first before this PR is merged. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Type of change Bug fix Testbed and Framework(new/improvement) Test case(new/improvement) Back port request 202012 202205 202305 202311 202405 Approach What is the motivation for this PR? This PR tracks only the infrastructure related changes needed for addition of the new testcases. How did you do it? Important changes are listed below: Change directory - tests/common/snappi_tests/ Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs. Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used. Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used. The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return. A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test). Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed. At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key. Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval. From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test. How did you verify/test it? Test was executed on the local clone. Any platform specific information? These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms. co-authorized by: jianquanye@microsoft.com

Description of PR This pull-request has changes specifically for the following commit-IDs: a82b489 180af4d 3da40bc This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Approach What is the motivation for this PR? Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode). How did you do it? Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps. This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports. Testcases: a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py: -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN. b. tests/snappi_tests/pfc/test_pfc_port_congestion.py: -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities. c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py: -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress. How did you verify/test it? Test case was executed on local clone. Results of the verification: Test cases executed for 100Gbps interfaces. Two combinations - single-line-card-multi-asic and multiple-dut Non-congestion: 19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic 19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv PASSED [ 16%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic 19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv PASSED [ 33%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic 19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic 19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv PASSED [ 66%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic 19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv PASSED [ 83%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic 19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv PASSED [100%] Over-subscription: 20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic 20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv PASSED [ 12%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic 20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv PASSED [ 25%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic 20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv PASSED [ 37%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic 20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic 21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv PASSED [ 62%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic 21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv PASSED [ 75%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic 21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv PASSED [ 87%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic 21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv PASSED [100%] PFCWD: 01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv PASSED [ 10%] 01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv PASSED [ 20%] 01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv PASSED [ 30%] 01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv PASSED [ 40%] 01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv PASSED [ 50%] 02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv PASSED [ 60%] 02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv PASSED [ 70%] 02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv PASSED [ 80%] 02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv PASSED [ 90%] 02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv PASSED [100%] Any platform specific information? The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT. co-authorized by: jianquanye@microsoft.com

[Snappi]: PFC - Mixed Speed testcases (sonic-net#14122) Description of PR As part of the new testcases to be added for the PFC-ECN, this PR addresses the mixed-speed ingress and egress testcases. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Approach What is the motivation for this PR? This script addresses the mixed speed testcases. The topology has single ingress and egress of 400Gbps and 100Gbps respectively. The congestion is caused due to three factors: Due to oversubscription of egress. Pause frames received on egress link of 100Gbps. Both - over-subscription of egress and pause frames received on egress. Idea is to test behavior of the DUT in these conditions. How did you do it? The port_map defines to choose single ingress of 400Gbps and egress of 100Gbps. Following test functions are used: test_mixed_speed_diff_dist_dist_over: Lossless and lossy traffic are sent at 88 and 12% of the line-rate (400Gbps) respectively, causing normal congestion on DUT due to oversubscription of the egress. Lossless priority 3 and 4 are used, whereas lossy priorities are 0,1 and 2. Expectation is that lossless priorities will cause DUT to send PAUSE frames to IXIA transmitter, will be rate-limited and hence no drops. Lossy priority traffic will see no drops at all. Egress throughput is expected to be around 100Gbps. Lossy ingress and egress throughput does not change. test_mixed_speed_uni_dist_dist_over: Lossless and lossy traffic are sent at 20% of the line-rate (400Gbps) respectively, causing normal congestion on DUT due to oversubscription of the egress. Lossless priority 3 and 4 are used, whereas lossy priorities are 0,1 and 2. Expectation is that lossless priorities will cause DUT to send PAUSE frames to IXIA transmitter, will be rate-limited and hence no drops. Lossy priority traffic will however see partial drop. Egress throughput is expected to be around 100Gbps with lossless and lossy traffic of equal (or close to equal) ratio. test_mixed_speed_pfcwd_enable: Lossless and lossy traffic are sent at 20% of the line-rate (400Gbps) respectively, causing normal congestion on DUT due to oversubscription of the egress. Lossless priority 3 and 4 are used, whereas lossy priorities are 0,1 and 2. Additionally, the IXIA receiver is sending PAUSE frames to DUT for lossless priority traffic. This causes additional congestion on the DUT. Expectation is that DUT sends PFC to the IXIA transmitter for lossless priorities in response to natural congestion on DUT due to oversubscription of egress. Lossless priority is rate-limited by IXIA in response to PFCs from DUT. Lossy priority is partially dropped on DUT. But since the DUT is receiving PFCs on egress, the rate-limited lossless traffic is eventually dropped on egress. The IXIA receiver receives ONLY 60Gbps of lossy traffic. test_mixed_speed_pfcwd_disable: Lossless and lossy traffic are sent at 20% of the line-rate (400Gbps) respectively, causing normal congestion on DUT due to oversubscription of the egress. Lossless priority 3 and 4 are used, whereas lossy priorities are 0,1 and 2. Additionally, the IXIA receiver is sending PAUSE frames to DUT for lossless priority traffic. This causes additional congestion on the DUT. Since PFCWD is disabled in this scenario, DUT forwards both lossless and lossy traffic to the IXIA receiver. DUT is sending PFCs in response to natural congestion as well as PFCs received on the egress. The egress line-rate is 100Gbps with lossy traffic being partially dropped. Lossy and lossless traffic are in equal (or close to equal) ratio. test_mixed_speed_no_congestion: Purpose of the testcase is to see if the DUT does not congestion in case the ingress 400Gbps is receiving 100Gbps of traffic, which it seamlessly moves to the egress without any drops or congestion. For all the above testcases, an additional check for the fabric counters is added. The tests will clear the fabric counters on line-cards and supervisor card (if part of the test). At the end of the test, counters are being checked again for CRC and uncorrectable FEC errors and asserts if the counts are non-zero. The checks are added as part of a different PR process and will need to be merged first. The underlying infra also needs to be added first before the testcases are added. How did you verify/test it? Tested on local platform. 16:05:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_400Gbps_Ingress_Single_100Gbps_Egress_diff_dist__multiple-dut-mixed-speed_1024B-2024-10-09-16-05.csv PASSED [ 20%] 16:13:48 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_400Gbps_Ingress_Single_100Gbps_Egress_uni_dist__multiple-dut-mixed-speed_1024B-2024-10-09-16-13.csv PASSED [ 40%] 16:22:13 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_400Gbps_Ingress_Single_100Gbps_Egress_pause_pfcwd_enable__multiple-dut-mixed-speed_1024B-2024-10-09-16-22.csv PASSED [ 60%] 16:30:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_400Gbps_Ingress_Single_100Gbps_Egress_pause_pfcwd_disable__multiple-dut-mixed-speed_1024B-2024-10-09-16-30.csv PASSED [ 80%] 16:38:56 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_400Gbps_Ingress_Single_100Gbps_Egress_no_cong__multiple-dut-mixed-speed_1024B-2024-10-09-16-38.csv PASSED [100%] Any platform specific information? The test is specifically meant for Broadcom-DNX multi-ASIC platforms ONLY. co-authorized by: jianquanye@microsoft.com

[Snappi] New testcases for PFC-ECN. (sonic-net#13865) Description of PR This pull-request has changes specifically for the following commit-IDs: a82b489 180af4d 3da40bc This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Approach What is the motivation for this PR? Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode). How did you do it? Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps. This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports. Testcases: a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py: -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN. b. tests/snappi_tests/pfc/test_pfc_port_congestion.py: -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities. c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py: -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress. How did you verify/test it? Test case was executed on local clone. Results of the verification: Test cases executed for 100Gbps interfaces. Two combinations - single-line-card-multi-asic and multiple-dut Non-congestion: 19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic 19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv PASSED [ 16%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic 19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv PASSED [ 33%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic 19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic 19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv PASSED [ 66%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic 19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv PASSED [ 83%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic 19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv PASSED [100%] Over-subscription: 20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic 20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv PASSED [ 12%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic 20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv PASSED [ 25%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic 20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv PASSED [ 37%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic 20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic 21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv PASSED [ 62%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic 21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv PASSED [ 75%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic 21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv PASSED [ 87%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic 21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv PASSED [100%] PFCWD: 01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv PASSED [ 10%] 01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv PASSED [ 20%] 01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv PASSED [ 30%] 01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv PASSED [ 40%] 01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv PASSED [ 50%] 02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv PASSED [ 60%] 02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv PASSED [ 70%] 02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv PASSED [ 80%] 02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv PASSED [ 90%] 02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv PASSED [100%] Any platform specific information? The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT. co-authorized by: jianquanye@microsoft.com

[Snappi] Infra changes for new PFC-ECN testcases. (sonic-net#13864) Description of PR 67989d1312b1778681d6575b12b66aa42fdf05a7 Please review the commit-ID given above. Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process. This PR tracks are the infrastructure related changes required for the execution of the testcases. Note - PR sonic-net#13848 needs to be merged in first before this PR is merged. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Approach What is the motivation for this PR? This PR tracks only the infrastructure related changes needed for addition of the new testcases. How did you do it? Important changes are listed below: Change directory - tests/common/snappi_tests/ Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs. Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used. Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used. The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return. A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test). Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed. At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key. Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval. From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test. How did you verify/test it? Test was executed on the local clone. Any platform specific information? These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms. co-authorized by: jianquanye@microsoft.com

Description of PR This pull-request has changes specifically for the following commit-IDs: a82b489 180af4d 3da40bc This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Approach What is the motivation for this PR? Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode). How did you do it? Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps. This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports. Testcases: a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py: -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN. b. tests/snappi_tests/pfc/test_pfc_port_congestion.py: -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities. c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py: -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress. How did you verify/test it? Test case was executed on local clone. Results of the verification: Test cases executed for 100Gbps interfaces. Two combinations - single-line-card-multi-asic and multiple-dut Non-congestion: 19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic 19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv PASSED [ 16%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic 19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv PASSED [ 33%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic 19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic 19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv PASSED [ 66%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic 19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv PASSED [ 83%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic 19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv PASSED [100%] Over-subscription: 20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic 20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv PASSED [ 12%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic 20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv PASSED [ 25%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic 20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv PASSED [ 37%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic 20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic 21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv PASSED [ 62%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic 21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv PASSED [ 75%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic 21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv PASSED [ 87%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic 21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv PASSED [100%] PFCWD: 01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv PASSED [ 10%] 01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv PASSED [ 20%] 01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv PASSED [ 30%] 01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv PASSED [ 40%] 01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv PASSED [ 50%] 02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv PASSED [ 60%] 02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv PASSED [ 70%] 02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv PASSED [ 80%] 02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv PASSED [ 90%] 02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv PASSED [100%] Any platform specific information? The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT. co-authorized by: jianquanye@microsoft.com

Description of PR 67989d1312b1778681d6575b12b66aa42fdf05a7 Please review the commit-ID given above. Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process. This PR tracks are the infrastructure related changes required for the execution of the testcases. Note - PR sonic-net#13848 needs to be merged in first before this PR is merged. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Type of change Bug fix Testbed and Framework(new/improvement) Test case(new/improvement) Back port request 202012 202205 202305 202311 202405 Approach What is the motivation for this PR? This PR tracks only the infrastructure related changes needed for addition of the new testcases. How did you do it? Important changes are listed below: Change directory - tests/common/snappi_tests/ Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs. Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used. Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used. The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return. A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test). Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed. At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key. Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval. From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test. How did you verify/test it? Test was executed on the local clone. Any platform specific information? These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms. co-authorized by: jianquanye@microsoft.com

Description of PR This pull-request has changes specifically for the following commit-IDs: a82b489 180af4d 3da40bc This PR specifically handles the testcases pertaining to the new PFC-ECN testplan added. Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215 Approach What is the motivation for this PR? Three test-scripts have been added to specifically test: non-congestion scenarios (line-rate tests), congestion testcases via over-subscription and PFCWD (drop and forward mode). How did you do it? Test case has dictionary called test_def which defines various testcases parameters necessary to run the testcase. An example of this, is packet-size (default is IMIX but can be changed to 1024B), test-duration, stats capture, file log at the end of the test. Similarly, there is test_check which passes test-case uses for verification info. Lossless and lossy priorities are selected from the available list. Most important change comes in form of port_map definition. Port map is a list with first two parameters defining the egress port count and egress speed. Last two parameters define the ingress port count and ingress speed. Example - [1, 100, 2 , 100] defines single egress port of speed 100Gbps and 2 ingress ports of 100Gbps. This definition is important because, multi-speed ingress and egress ports needs to be supported. Example - [1, 100, 1, 400] will define single ingress and egress of 400Gbps and 100Gbps respectively. A new function is provided to capture snappi_ports. This will pick the line-card choice from variable.py and choose the ports as defined in port_map. The port_map is used to filter out the available ports for the required port-speed. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. Additional checks are present in multi_dut helper file, depending upon the type of the test. The test passes the verification parameters in test_check in dictionary format. There is important change in variables.py file. The line_card_choice is sent as dictionary from variables.py, which then is parameterized in the test. Depending upon the type of line_card_choice, the tests are ran for that specific line_card choice and set of ports. Testcases: a. tests/snappi_tests/pfc/test_pfc_no_congestion_throughput.py: -- This testcase has testcases to test line-rate speeds with single ingress and egress. Traffic combination around lossless and lossy priorities have been used. Expectations is that no PFCs will be generated, line-rate will be achieved, no drops will be seen on both DUT and TGEN. b. tests/snappi_tests/pfc/test_pfc_port_congestion.py: -- This testcase has testcases to test behavior with 2 ingress ports and 1 egress port on the DUT. Traffic combination around lossless and lossy priorities. c. tests/snappi_tests/pfcwd/test_pfcwd_actions.py: -- Testcases cover PFCWD action - DROP and FORWARD mode. DROP and FORWARD mode is also tested for two ingresses and single egress with pause frames on egress. How did you verify/test it? Test case was executed on local clone. Results of the verification: Test cases executed for 100Gbps interfaces. Two combinations - single-line-card-multi-asic and multiple-dut Non-congestion: 19:06:48 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-multi-asic 19:15:21 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-15.csv PASSED [ 16%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 19:15:26 test_sys_non_congestion.test_multiple_pr L0095 INFO | Running test for testbed subtype: single-dut-single-asic 19:23:37 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-23.csv PASSED [ 33%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 19:23:42 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-multi-asic 19:31:57 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-31.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 19:32:02 test_sys_non_congestion.test_multiple_pr L0235 INFO | Running test for testbed subtype: single-dut-single-asic 19:40:12 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_uni_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-40.csv PASSED [ 66%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info0-port_map0] 19:40:18 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-multi-asic 19:48:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-multi-asic_1024B-2024-10-08-19-48.csv PASSED [ 83%] snappi_tests/multidut/systest/test_sys_non_congestion.py::test_single_lossless_prio[multidut_port_info1-port_map0] 19:48:31 test_sys_non_congestion.test_single_loss L0375 INFO | Running test for testbed subtype: single-dut-single-asic 19:56:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Egress_1Prio_linerate_100Gbps_single-dut-single-asic_1024B-2024-10-08-19-56.csv PASSED [100%] Over-subscription: 20:13:40 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-multi-asic 20:23:07 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-23.csv PASSED [ 12%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_diff_dist[multidut_port_info1-port_map0] 20:23:16 test_sys_over_subscription.test_multiple L0093 INFO | Running test for testbed subtype: single-dut-single-asic 20:32:20 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_diff_dist_100Gbps_single-dut-single-asic_1024B-2024-10-08-20-32.csv PASSED [ 25%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info0-port_map0] 20:32:29 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-multi-asic 20:41:39 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-20-41.csv PASSED [ 37%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist[multidut_port_info1-port_map0] 20:41:48 test_sys_over_subscription.test_multiple L0227 INFO | Running test for testbed subtype: single-dut-single-asic 20:50:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-20-50.csv PASSED [ 50%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info0-port_map0] 20:51:02 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-multi-asic 21:00:11 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-00.csv PASSED [ 62%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_uni_dist_full[multidut_port_info1-port_map0] 21:00:20 test_sys_over_subscription.test_multiple L0364 INFO | Running test for testbed subtype: single-dut-single-asic 21:09:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_uni_dist_full100Gbps_single-dut-single-asic_1024B-2024-10-08-21-09.csv PASSED [ 75%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info0-port_map0] 21:09:34 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-multi-asic 21:18:38 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-08-21-18.csv PASSED [ 87%] snappi_tests/multidut/systest/test_sys_over_subscription.py::test_multiple_prio_non_cngtn[multidut_port_info1-port_map0] 21:18:47 test_sys_over_subscription.test_multiple L0502 INFO | Running test for testbed subtype: single-dut-single-asic 21:27:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_non_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-08-21-27.csv PASSED [100%] PFCWD: 01:08:43 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-08.csv PASSED [ 10%] 01:19:33 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_drop_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-19.csv PASSED [ 20%] 01:30:32 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-30.csv PASSED [ 30%] 01:41:25 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/One_Ingress_Egress_pfcwd_frwd_90_10_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-01-41.csv PASSED [ 40%] 01:53:08 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-01-53.csv PASSED [ 50%] 02:04:49 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_drop_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-04.csv PASSED [ 60%] 02:16:26 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-16.csv PASSED [ 70%] 02:27:53 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Two_Ingress_Single_Egress_pfcwd_frwd_40_9_dist100Gbps_single-dut-single-asic_1024B-2024-10-09-02-27.csv PASSED [ 80%] 02:38:45 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-multi-asic_1024B-2024-10-09-02-38.csv PASSED [ 90%] 02:49:22 traffic_generation.run_sys_traffic L1190 INFO | Writing statistics to file : /tmp/Single_Ingress_Single_Egress_pause_cngstn_100Gbps_single-dut-single-asic_1024B-2024-10-09-02-49.csv PASSED [100%] Any platform specific information? The testcases are specifically meant for Broadcom DNX Multi-ASIC platform DUT. co-authorized by: jianquanye@microsoft.com

amitpawar12 · 2025-03-19T17:52:24Z

Closing this pull-request as this is already taken care by following pull-request #13864

amitpawar12 force-pushed the snappi-pfc-new-testcase-infra branch from b0526c5 to ff2ae4e Compare July 16, 2024 20:07

sdszhang added the Request for 202405 branch label Jul 18, 2024

amitpawar12 added 3 commits July 22, 2024 15:29

PFC-ECN-Snappi-New-Systest-PR-13215

2b3b7f0

Snappi:Added new over-subscription cases, fixed paths, PFC checks

2ea9c7e

PR-13490-New-Snappi-PFC testcase for PFCWD

561b2c5

amitpawar12 force-pushed the snappi-pfc-new-testcase-infra branch from ff2ae4e to 561b2c5 Compare July 22, 2024 19:47

sdszhang reviewed Jul 24, 2024

View reviewed changes

This was referenced Jul 29, 2024

[Snappi] Infra changes for new PFC-ECN testcases. #13864

Merged

[Snappi] New testcases for PFC-ECN. #13865

Merged

vmittal-msft self-assigned this Aug 2, 2024

vmittal-msft requested review from developfast and vmittal-msft August 2, 2024 00:16

amitpawar12 mentioned this pull request Aug 14, 2024

[Snappi]: PFC - Mixed Speed testcases #14122

Merged

8 tasks

veronica-arista reviewed Sep 11, 2024

View reviewed changes

mssonicbld mentioned this pull request Jan 14, 2025

[action] [PR:13864] [Snappi] Infra changes for new PFC-ECN testcases. #16505

Merged

8 tasks

mssonicbld mentioned this pull request Jan 15, 2025

[action] [PR:14122] [Snappi]: PFC - Mixed Speed testcases #16524

Closed

8 tasks

mssonicbld mentioned this pull request Jan 15, 2025

[action] [PR:13865] [Snappi] New testcases for PFC-ECN. #16525

Merged

8 tasks

mssonicbld mentioned this pull request Jan 16, 2025

[action] [PR:14122] [Snappi]: PFC - Mixed Speed testcases #16538

Merged

8 tasks

rlhui assigned amitpawar12 and unassigned vmittal-msft Jan 29, 2025

amitpawar12 closed this Mar 19, 2025

github-project-automation bot moved this to Done in SONiC Chassis Mar 19, 2025

Conversation

amitpawar12 commented Jul 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

sdszhang commented Jul 18, 2024

Uh oh!

sdszhang commented Jul 24, 2024

Uh oh!

sdszhang Jul 24, 2024

Choose a reason for hiding this comment

Uh oh!

sdszhang Jul 24, 2024

Choose a reason for hiding this comment

Uh oh!

sdszhang Jul 24, 2024

Choose a reason for hiding this comment

Uh oh!

yxieca commented Jul 24, 2024

Uh oh!

amitpawar12 commented Jul 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sdszhang commented Jul 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sdszhang commented Jul 24, 2024

Uh oh!

developfast commented Jul 24, 2024

Uh oh!

amitpawar12 commented Jul 25, 2024

Uh oh!

developfast commented Jul 25, 2024

Uh oh!

amitpawar12 commented Jul 26, 2024

Uh oh!

amitpawar12 commented Jul 29, 2024

Uh oh!

veronica-arista Sep 11, 2024

Choose a reason for hiding this comment

Uh oh!

amitpawar12 commented Mar 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

amitpawar12 commented Jul 14, 2024 •

edited

Loading

amitpawar12 commented Jul 24, 2024 •

edited

Loading

sdszhang commented Jul 24, 2024 •

edited

Loading