Skip to content

[Snappi] Infra changes for new PFC-ECN testcases.#13864

Merged
yejianquan merged 11 commits intosonic-net:masterfrom
amitpawar12:Snappi-Base-PR-13655-Infra
Jan 14, 2025
Merged

[Snappi] Infra changes for new PFC-ECN testcases.#13864
yejianquan merged 11 commits intosonic-net:masterfrom
amitpawar12:Snappi-Base-PR-13655-Infra

Conversation

@amitpawar12
Copy link
Contributor

@amitpawar12 amitpawar12 commented Jul 29, 2024

Description of PR

67989d1312b1778681d6575b12b66aa42fdf05a7

Please review the commit-ID given above.

Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process.

This PR tracks are the infrastructure related changes required for the execution of the testcases.

Note - PR #13848 needs to be merged in first before this PR is merged.

Summary:
Fixes # (issue)
#13655
#13215

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 202012
  • 202205
  • 202305
  • 202311
  • 202405

Approach

What is the motivation for this PR?

This PR tracks only the infrastructure related changes needed for addition of the new testcases.

How did you do it?

Important changes are listed below:

Change directory - tests/common/snappi_tests/

  1. Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs.

  2. Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used.

  3. Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used.

  4. The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return.

  5. A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test).

  6. Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
    At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.

Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request.

  1. At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test.

How did you verify/test it?

Test was executed on the local clone.

Any platform specific information?

These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms.

Supported testbed topology if it's a new test case?

Documentation

@sdszhang
Copy link
Contributor

@rraghav-cisco for review.

@amitpawar12 amitpawar12 force-pushed the Snappi-Base-PR-13655-Infra branch from 78c1829 to abbb216 Compare August 28, 2024 21:21
@rlhui rlhui assigned amitpawar12 and unassigned vmittal-msft Sep 1, 2024
@rraghav-cisco
Copy link
Contributor

@rraghav-cisco for review.

@sdszhang : Verified the diff works in my workspace. Approved.

@amitpawar12 amitpawar12 force-pushed the Snappi-Base-PR-13655-Infra branch from abbb216 to 2d5673e Compare October 7, 2024 18:48
ixnetwork(obj): rest_py API object.
"""

api = snappi.api(location="1.1.30.1", ext="ixnetwork")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get IP and credential from existing infra code instead of hardcode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I think that can be part of a different PR to address this. IXIA IP-address and credentials, both need to be pulled from existing infra-code.

from tests.common.snappi_tests.multi_dut_params import MultiDUTParams


class SnappiSysTestParams():
Copy link
Contributor

@sdszhang sdszhang Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a duplication of SnappiTestParams? Can we reuse SnappiTestParams?
If this is needed, can we rename it for easier understanding. e.g.: SnappiTrafficGroupParams, SnappiFlowBuilderParams, or something similar?

if port["peer_port"] in asic_port_map[asic] and hostname in port['peer_device']:
port['asic_value'] = asic
port['asic_type'] = host.facts["asic_type"]
port['duthost'] = host
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this change needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sdszhang - I thought this change came from @selldinesh and is not my change. We can discuss this and check if this is required or not.

Copy link
Contributor

@sdszhang sdszhang Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this change is not required. We will need another PR to cleanup unused functions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amitpawar12 can you remove this change from this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sdszhang, This code is still part of the code and I am not sure why it was added in first place. I would recommend that we raise a different PR to remove this line of the code. I am not sure why it was added, and impact in case we removed it.

Thanks,

avg_latency = round(df_t[item].mean(), 2)
if (flow in item and item.split(flow)[1] == '_rx_pkts'):
rx_pkts = df_t[item].max()
if (flow in item and item.split(flow)[1] == '_rx_pkts' and ('prio_3' in flow or 'prio_4' in flow)):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can use the lossless priority list and lossy priority list instead of hard code prio_3/prio_4.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amitpawar12 what's the plan to resolve this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sdszhang - This is fixed now.

@amitpawar12
Copy link
Contributor Author

@sdszhang , @developfast

For the port_map, this is going to be the format, that will be used to parameterize:
port_map = [{'egress_speed': 400, 'egress_num_link': 1, 'ingress_speed': 400, 'ingress_num_link': 1},
{'egress_speed': 100, 'egress_num_link': 1, 'ingress_speed': 100, 'ingress_num_link': 1}]

Internal test will see it as dictionary. Please let me know so that I can start working on it.

@developfast
Copy link
Contributor

developfast commented Oct 10, 2024

Can we set it up as a dataclass @amitpawar12 for example,

from dataclasses import dataclass
from typing import List

@dataclass
class PortConfig:
    egress_speed: int
    egress_num_link: int
    ingress_speed: int
    ingress_num_link: int
   .....(add more attributes as necessary)

# Define the port map as a list of PortConfig objects
port_map: List[PortConfig] = [
    PortConfig(egress_speed=400, egress_num_link=1, ingress_speed=400, ingress_num_link=1),
    PortConfig(egress_speed=100, egress_num_link=1, ingress_speed=100, ingress_num_link=1)
]

# Now you can access the port_map elements as objects
print(port_map)

This is cleaner and we can keep track of keys and their types. It also ensures we don't "accidentally" add new keys without first adding it to the dataclass

@amitpawar12
Copy link
Contributor Author

Thanks for the comments @developfast.

The port_map was an interim solution to ensure that we can run tests on both 100 and 400Gbps line-cards together. However, ideal scenario would be get rid of variables.py and pick up the ports (both 100 and 400Gbps, and of any other speeds) and make it available for the test.

I have PR for the same:
#15069

Thanks,
-A

@amitpawar12 amitpawar12 force-pushed the Snappi-Base-PR-13655-Infra branch from 2d5673e to a031921 Compare October 30, 2024 15:54
@amitpawar12 amitpawar12 force-pushed the Snappi-Base-PR-13655-Infra branch from a031921 to 6a99744 Compare December 2, 2024 17:40
@amitpawar12 amitpawar12 force-pushed the Snappi-Base-PR-13655-Infra branch from ba465d8 to 4f8480b Compare December 10, 2024 20:52
@amitpawar12 amitpawar12 force-pushed the Snappi-Base-PR-13655-Infra branch from 4f8480b to 29c5e36 Compare December 11, 2024 17:08
@sdszhang
Copy link
Contributor

@amitpawar12 can you update the test results.

@amitpawar12
Copy link
Contributor Author

@amitpawar12 can you update the test results.

Logs are kept at following link:
#13865 (comment)

Thanks,
-A

@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@sdszhang
Copy link
Contributor

@selldinesh can you help to upload the T0/T1 test result?

@developfast for viz.

@amitpawar12 amitpawar12 force-pushed the Snappi-Base-PR-13655-Infra branch from 0d67ff5 to f64b85c Compare January 10, 2025 21:54
@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@sdszhang
Copy link
Contributor

Good to merge to master and msft.202405. No need for 202405.

@yejianquan yejianquan merged commit 4f28bfe into sonic-net:master Jan 14, 2025
mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Jan 14, 2025
Description of PR
67989d1312b1778681d6575b12b66aa42fdf05a7

Please review the commit-ID given above.

Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process.

This PR tracks are the infrastructure related changes required for the execution of the testcases.

Note - PR sonic-net#13848 needs to be merged in first before this PR is merged.

Summary:
Fixes # (issue)
sonic-net#13655
sonic-net#13215

Type of change
 Bug fix
 Testbed and Framework(new/improvement)
 Test case(new/improvement)
Back port request
 202012
 202205
 202305
 202311
 202405
Approach
What is the motivation for this PR?
This PR tracks only the infrastructure related changes needed for addition of the new testcases.

How did you do it?
Important changes are listed below:

Change directory - tests/common/snappi_tests/

Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs.

Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used.

Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used.

The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return.

A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test).

Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.

Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request.

At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test.
How did you verify/test it?
Test was executed on the local clone.

Any platform specific information?
These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms.

co-authorized by: [email protected]
@mssonicbld
Copy link
Collaborator

Cherry-pick PR to 202411: #16505

mssonicbld pushed a commit that referenced this pull request Jan 15, 2025
Description of PR
67989d1312b1778681d6575b12b66aa42fdf05a7

Please review the commit-ID given above.

Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process.

This PR tracks are the infrastructure related changes required for the execution of the testcases.

Note - PR #13848 needs to be merged in first before this PR is merged.

Summary:
Fixes # (issue)
#13655
#13215

Type of change
 Bug fix
 Testbed and Framework(new/improvement)
 Test case(new/improvement)
Back port request
 202012
 202205
 202305
 202311
 202405
Approach
What is the motivation for this PR?
This PR tracks only the infrastructure related changes needed for addition of the new testcases.

How did you do it?
Important changes are listed below:

Change directory - tests/common/snappi_tests/

Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs.

Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used.

Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used.

The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return.

A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test).

Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.

Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request.

At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test.
How did you verify/test it?
Test was executed on the local clone.

Any platform specific information?
These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms.

co-authorized by: [email protected]
yejianquan pushed a commit to yejianquan/sonic-mgmt that referenced this pull request Jan 17, 2025
Description of PR
67989d1312b1778681d6575b12b66aa42fdf05a7

Please review the commit-ID given above.

Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process.

This PR tracks are the infrastructure related changes required for the execution of the testcases.

Note - PR sonic-net#13848 needs to be merged in first before this PR is merged.

Summary:
Fixes # (issue)
sonic-net#13655
sonic-net#13215

Type of change
 Bug fix
 Testbed and Framework(new/improvement)
 Test case(new/improvement)
Back port request
 202012
 202205
 202305
 202311
 202405
Approach
What is the motivation for this PR?
This PR tracks only the infrastructure related changes needed for addition of the new testcases.

How did you do it?
Important changes are listed below:

Change directory - tests/common/snappi_tests/

Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs.

Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used.

Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used.

The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return.

A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test).

Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.

Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request.

At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test.
How did you verify/test it?
Test was executed on the local clone.

Any platform specific information?
These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms.

co-authorized by: [email protected]
yejianquan added a commit to yejianquan/sonic-mgmt that referenced this pull request Jan 17, 2025
[Snappi] Infra changes for new PFC-ECN testcases. (sonic-net#13864)

Description of PR
67989d1312b1778681d6575b12b66aa42fdf05a7

Please review the commit-ID given above.

Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process.

This PR tracks are the infrastructure related changes required for the execution of the testcases.

Note - PR sonic-net#13848 needs to be merged in first before this PR is merged.

Summary: Fixes # (issue) sonic-net#13655 sonic-net#13215

Approach
What is the motivation for this PR?
This PR tracks only the infrastructure related changes needed for addition of the new testcases.

How did you do it?
Important changes are listed below:

Change directory - tests/common/snappi_tests/

Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs.
Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used.
Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used.
The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return.
A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test).
Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.
Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request.

At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test.
How did you verify/test it?
Test was executed on the local clone.

Any platform specific information?
These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms.

co-authorized by: [email protected]
wangxin pushed a commit to wangxin/sonic-mgmt that referenced this pull request Feb 21, 2025
Description of PR
67989d1312b1778681d6575b12b66aa42fdf05a7

Please review the commit-ID given above.

Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process.

This PR tracks are the infrastructure related changes required for the execution of the testcases.

Note - PR sonic-net#13848 needs to be merged in first before this PR is merged.

Summary:
Fixes # (issue)
sonic-net#13655
sonic-net#13215

Type of change
 Bug fix
 Testbed and Framework(new/improvement)
 Test case(new/improvement)
Back port request
 202012
 202205
 202305
 202311
 202405
Approach
What is the motivation for this PR?
This PR tracks only the infrastructure related changes needed for addition of the new testcases.

How did you do it?
Important changes are listed below:

Change directory - tests/common/snappi_tests/

Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs.

Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used.

Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used.

The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return.

A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test).

Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.

Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request.

At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test.
How did you verify/test it?
Test was executed on the local clone.

Any platform specific information?
These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms.

co-authorized by: [email protected]
wangxin pushed a commit to wangxin/sonic-mgmt that referenced this pull request Feb 21, 2025
Merge 202405 branch in as of 12:23pm 20/01/2025 AEST

b118611 (HEAD -> merge/202405) Use alternate check for reboot for T2 after reboot with REBOOT_TYPE_POWEROFF (sonic-net#16348)
0e0e898 flap interface after sfp reset (sonic-net#16375)
41e2b2f Temporarily skip lpmode test for some transceivers with known issue (sonic-net#16547)
de60273 [Snappi] Infra changes for new PFC-ECN testcases. (sonic-net#13864)
7b357f5 [Snappi] New testcases for PFC-ECN. (sonic-net#13865)
3523a7f [Snappi]: PFC - Mixed Speed testcases (sonic-net#14122)
3754f2a sonic-mgmt: Fix namespace issues for qos tests on T2 single ASIC (sonic-net#15708)
21f6526 [sonic-net#16015 Fix]: Cleaning up unused code from snappi_fixtures (sonic-net#16026)
d8f23be Correcting client arguments to dynamically_compensate_leakout (sonic-net#16169)
3c47107 [sanity_check][bgp] Enhance sanity check recover for bgp default route missing (sonic-net#16357)
37352b8 Eliminate cross-feature dependency from macsec module (sonic-net#15617)
4f33b0d (pub_upstream/202405) [202405][dhcp_relay] Add test case to verify dhcp6relay LLA waiting logic (sonic-net#16494) (sonic-net#16567)
nnelluri-cisco pushed a commit to nnelluri-cisco/sonic-mgmt that referenced this pull request Mar 15, 2025
Description of PR
67989d1312b1778681d6575b12b66aa42fdf05a7

Please review the commit-ID given above.

Original PR13655 was raised to add the new testcases. However, manage the changes efficiently, it was decided to split the original into three PRs for ease in review process.

This PR tracks are the infrastructure related changes required for the execution of the testcases.

Note - PR sonic-net#13848 needs to be merged in first before this PR is merged.

Summary:
Fixes # (issue)
sonic-net#13655
sonic-net#13215

Type of change
 Bug fix
 Testbed and Framework(new/improvement)
 Test case(new/improvement)
Back port request
 202012
 202205
 202305
 202311
 202405
Approach
What is the motivation for this PR?
This PR tracks only the infrastructure related changes needed for addition of the new testcases.

How did you do it?
Important changes are listed below:

Change directory - tests/common/snappi_tests/

Additional member variable 'base_flow_config_list' is added as list to class 'SnappiTestParams' in snappi_test_params.py file to accommodate for multiple base-flow-configs.

Existing functions - generate_test_flows, generate_background_flows, generate_pause_flows are modified to check if the base_flow_config_list exists. If it does, then base_flow_config is assigned snappi_extra_params.base_flow_config_list[flow_index]. Else existing code is used.

Existing function - 'verify_egress_queue_frame_count' is modified to check if base_flow_config_list exists. If yes, base_flow_config_list[0] is assigned to dut_port_config, else existing code is used.

The testcases calls 'run_traffic_and_collect_stats' function in traffic_generation file to run and gather IXIA+DUT statistics. Statistics are summarized in test_stats dictionary in return.

A function has been created to access the IXIA rest_py framework. This will in turn can be used to integrate MACSEC related changes in future. Currently, rest_py is used to generate the imix custom profile if the flag is set in the test_def dictionary (defined and passed by the test).

Depending upon the test_duration and test_interval defined in test_def of the test, the test-case will be executed.
At every test_interval, the statistics from IXIA and DUT are pulled in form of dictionary, where date-timestamp is primary key.

Important parameters from IXIA like Tx and Rx throughput, number of packets, latency etc are captured with each interval.

From DUT side, the Rx and Tx packets, loss packets (combination of failures, drops and errors), PFC count, queue counts are captured. Additional functions like - get_pfc_count, get_ingerface_stats etc are defined in the common/snappi_test helper files to assist with the same. The support for the above is added as part of the different pull-request.

At the end of the test, a CSV is created as raw data for the test-case execution. Summary of the test-case is generated in form of text file with same name. The run_sys_test also returns a dictionary test_stats with all the important parameters to be used for the verification of the test.
How did you verify/test it?
Test was executed on the local clone.

Any platform specific information?
These testcases are specifically meant for Broadcom-DNX multi-ASIC based platforms.

co-authorized by: [email protected]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

8 participants