[Spytest] Add multi-DUT testbed deployment support by lolyu · Pull Request #1848 · sonic-net/sonic-mgmt

lolyu · 2020-07-06T08:05:02Z

Description of PR

In the full-mesh topology used in Spytest, DUTs are directly-connected while the traffic generation
ports are connected through fanout switches with the use of PTF. This patch aims to automate the
process of the following steps:

start PTF container on the target server.
create vlan virtual devices on the target server.
inject the vlan virtual devices specified in the topology file into the PTF container created.
assign vlans to the trunk port connecting root fanout switch to the target server.

Signed-off-by: Longxiang Lyu lolv@microsoft.com

Summary:
Fixes # (issue)

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Approach

What is the motivation for this PR?

Automate the connection setup between DUTs and the PTF container.

How did you do it?

Add new topology fullmesh to support the definition of connections between multiple DUTs and the
PTF container.
- 0.1 means to connect the second port('1') from the first DUT('0').
testbed.csv should contain an entry like the following to describe the testbed.

spytest,spytest,fullmesh,<ptf_image>,ptf-unknown,<ptf_ip>,<server>,,[lab-msn2700-spy-01;lab-msn2700-spy-02;lab-msn2700-spy-03;lab-msn2700-spy-04],hellospytest

Add new argument hosts to conn_graph_facts ansible module to support parsing connection graph
for multiple DUTs.
- in this case, device_vlan_list will return a list of lists for the VLAN IDs of each DUT
  - device_vlan_list: [[201, 202, 203, 204], [205, 206, 207, 208], [209, 210, 211, 212], [213, 214, 215, 216]]
Modify the ansible module vlan_port inside role vm_set to create vlan ports based on the nested device_vlan_list.
- the created vlan ports will be returned, which will also be a nested list with results for each DUT
  - dut_fp_ports: [['p4p1.201', 'p4p1.202', 'p4p1.203', 'p4p1.204'], ['p4p1.205', 'p4p1.206', 'p4p1.207', 'p4p1.208'], ['p4p1.209', 'p4p1.210', 'p4p1.211', 'p4p1.212'], ['p4p1.213', 'p4p1.214', 'p4p1.215', 'p4p1.216']]
Modify the ansible module vm_topology to insert the vlan ports into the PTF container based on the nested dut_fp_ports and topo file.
- the host_interfaces inside topo file will be the indexes to choose the ports from dut_fp_ports.
  - 0.1 will choose the second port(p4p1.202) in the first DUT(['p4p1.201', 'p4p1.202', 'p4p1.203', 'p4p1.204'])

How did you verify/test it?

# ./testbed-cli.sh add-topo spytest password.txt
# ./testbed-cli.sh remove-topo spytest password.txt

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

tahmed-dev · 2020-07-07T18:51:25Z

ansible/library/conn_graph_facts.py

Wonder how the vlan used to connect DUTs are configured. If DUT1 has port 0 connected to DUT2 port 0, then both should have the same vlan id. Also, vlans in trunk interfaces all the way up to the server will need to be updated.

The ports between DUTs are direct connections, this patch aims to connect PTF containers through fanout switches. The VLAN trunk interfaces connecting to the server is configured in playbook fanout_connect.yml.

ansible/fanout_connect.yml

'fullmesh' describes the topology used for sytest, which supports multi-DUTs definition. `"0.1"` means connecting the second port('1') of the first DUT('0') to the PTF container. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

lolyu · 2020-07-08T15:04:50Z

@yxieca please help review, thanks!

yxieca · 2020-07-08T16:57:11Z

Please fill in the required information as much as you have:

Approach
What is the motivation for this PR?
How did you do it?
How did you verify/test it?
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation

lolyu · 2020-07-09T03:55:43Z

Please fill in the required information as much as you have:

Approach
What is the motivation for this PR?
How did you do it?
How did you verify/test it?
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation

Updated, thanks.

ansible/library/conn_graph_facts.py

1. Add an argument `hosts` to pass a list of DUTs to the module. 2. When the inquiring target is a list of DUTs, make the returns nested consisting of the corresponding return for each DUT. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

1. Modified module `vlan_port` to support multiple DUTs that will return `dut_fp_ports` consisting of lists of vlan virtual devices for each DUT. 2. Modified module `vm_topology` to support multiple DUTs that it will inject the vlan virtual devices(listed in `dut_fp_ports`, specified by the `host_interfaces` in topology file) into PTF container. 3. Other changes to adapt to that `dut_name` might be a string of multiple DUTs separated by comma. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

Modified `rootfanout_connect.yml` to support assigning vlans from multiple DUTs to the root fanout trunk port that is connected to the vm server. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

lolyu · 2020-07-15T07:09:29Z

@yxieca, I've made some changes that before calling conn_graph_facts, check if dut_name contains a single DUT or multiple DUTs to choose either host or hosts to use.

d29a49a [ACL] Match TCP protocol while matching TCP_FLAG (sonic-net#1854) 2569ad9 Fix sFlow sampling-rate and admin-state (sonic-net#1728) 8908a8f Change rif_rates.lua and port_rates.lua scripts to calculate rates correct (sonic-net#1848) b42c2fb [VS Test] Skip flaky tests (sonic-net#1875) Signed-off-by: bingwang <bingwang@microsoft.com>

* 647f01f 2021-09-30 | [fstrim] limit smartctl execution time to 30 seconds (sonic-net#1850) (HEAD -> 202012, github/202012) [Ying Xie] * 371970e 2021-09-30 | [202012][show techsupport] address show techsupport return none zero code issue (sonic-net#1847) [Ying Xie] * 81a8386 2021-09-30 | Disable pfcwd forward action config for cisco-8000. (sonic-net#1848) [Alpesh Patel] Signed-off-by: Ying Xie <ying.xie@microsoft.com>

88a38f7 Ignore ALREADY_EXIST error in FDB creation (sonic-net#1815) b1c23f3 Change rif_rates.lua and port_rates.lua scripts to calculate rates correct (sonic-net#1848) Update sonic-utilities submodule with cbc25d6 [config reload] Call systemctl reset-failed for snmp,telemetry,mgmt-framework services (sonic-net#1773) 04dcd07 Improve config error handling on version_info (sonic-net#1760) e567a60 Load the database global_db. (sonic-net#1752) c15fb8f [sfputil] Gracefully handle improper 'specification_compliance' field (sonic-net#1741) 39350f8 [dhcp_relay] Update CLI reference document and add a new API for ip address type (sonic-net#1717) 18f13c6 [sonic-package-manager] switch from poetry-semver to semantic_version due to bugs found in poetry-semver (sonic-net#1710) b16724a [voq][chassis] VOQ cli show commands implementation (sonic-net#1689) 9427cd6 [debug dump util] Match Infrastructure (sonic-net#1666) d9fb39b [route_check] Filter out VNET routes (sonic-net#1612)

lolyu force-pushed the spytest_deploy branch from 7674ece to 850201f Compare July 6, 2020 08:27

yxieca requested a review from a team July 6, 2020 15:15

lolyu force-pushed the spytest_deploy branch 2 times, most recently from d229da2 to 3684895 Compare July 7, 2020 07:40

sonic-net deleted a comment from lgtm-com bot Jul 7, 2020

lolyu force-pushed the spytest_deploy branch 3 times, most recently from 4ca6401 to 5308c1b Compare July 7, 2020 13:50

tahmed-dev reviewed Jul 7, 2020

View reviewed changes

[ansible] Add new topo 'fullmesh'

8d7b581

'fullmesh' describes the topology used for sytest, which supports multi-DUTs definition. `"0.1"` means connecting the second port('1') of the first DUT('0') to the PTF container. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

lolyu force-pushed the spytest_deploy branch from 5308c1b to 35b2dda Compare July 8, 2020 07:38

lolyu marked this pull request as ready for review July 8, 2020 07:47

lolyu added Enhancement Spytest Testbed labels Jul 8, 2020

lolyu force-pushed the spytest_deploy branch from 35b2dda to 99b1751 Compare July 8, 2020 12:41

yxieca changed the title ~~[Spytest] Add testbed deployment support~~ [Spytest] Add multi-DUT testbed deployment support Jul 8, 2020

lolyu force-pushed the spytest_deploy branch from 99b1751 to 365a853 Compare July 13, 2020 01:43

yxieca reviewed Jul 13, 2020

View reviewed changes

ansible/library/conn_graph_facts.py Outdated Show resolved Hide resolved

lolyu force-pushed the spytest_deploy branch from f10c73a to 365a853 Compare July 14, 2020 02:12

lolyu force-pushed the spytest_deploy branch from 365a853 to 7a9616a Compare July 14, 2020 03:05

yxieca approved these changes Jul 14, 2020

View reviewed changes

lolyu added 2 commits July 14, 2020 05:54

[ansible] Support removing spytest testbed

9e00431

Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

lolyu force-pushed the spytest_deploy branch 2 times, most recently from 54d58a4 to 6006c2d Compare July 14, 2020 13:29

lolyu force-pushed the spytest_deploy branch 2 times, most recently from e51fe09 to 6bea4d8 Compare July 15, 2020 02:51

[ansible/fanout_connect.yml] Support the deployment of Spytest

ae87da1

Modified `rootfanout_connect.yml` to support assigning vlans from multiple DUTs to the root fanout trunk port that is connected to the vm server. Signed-off-by: Longxiang Lyu <lolv@microsoft.com>

lolyu force-pushed the spytest_deploy branch from 6bea4d8 to ae87da1 Compare July 15, 2020 05:33

yxieca approved these changes Jul 15, 2020

View reviewed changes

lolyu merged commit 6ef53cb into sonic-net:master Jul 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Spytest] Add multi-DUT testbed deployment support#1848

[Spytest] Add multi-DUT testbed deployment support#1848
lolyu merged 5 commits intosonic-net:masterfrom
lolyu:spytest_deploy

lolyu commented Jul 6, 2020 •

edited

Loading

Uh oh!

tahmed-dev Jul 7, 2020

Uh oh!

lolyu Jul 8, 2020 •

edited

Loading

Uh oh!

Uh oh!

lolyu commented Jul 8, 2020

Uh oh!

yxieca commented Jul 8, 2020

Uh oh!

lolyu commented Jul 9, 2020

Uh oh!

Uh oh!

lolyu commented Jul 15, 2020 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

lolyu commented Jul 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

tahmed-dev Jul 7, 2020

Choose a reason for hiding this comment

Uh oh!

lolyu Jul 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lolyu commented Jul 8, 2020

Uh oh!

yxieca commented Jul 8, 2020

Uh oh!

lolyu commented Jul 9, 2020

Uh oh!

Uh oh!

lolyu commented Jul 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lolyu commented Jul 6, 2020 •

edited

Loading

lolyu Jul 8, 2020 •

edited

Loading

lolyu commented Jul 15, 2020 •

edited

Loading