[Sub-ports] Added new sub-ports test cases by OleksandrKozodoi · Pull Request #2669 · sonic-net/sonic-mgmt

OleksandrKozodoi · 2020-12-15T13:24:41Z

Signed-off-by: Oleksandr Kozodoi [email protected]

Description of PR

Summary:

Added new sub-ports tests:

Updated Test Plan
Added support of LAG port
Added support of t1 topology
Changed pre-configuration of test cases
Updated teardown of test cases

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Approach

What is the motivation for this PR?

Coverage of sub-ports feature by test cases to improve the quality of SONiC.

How did you do it?

Added new test cases to PyTest.
Implementation is based on the Sub-ports design spec

How did you verify/test it?

py.test --testbed=testbed-t0 --inventory=../ansible/lab --testbed_file=../ansible/testbed.csv --host-pattern=testbed-t0 -- module-path=../ansible/library sub_port_interfaces

Any platform specific information?

SONiC Software Version: SONiC.master.130-dirty-20210221.030317
Distribution: Debian 10.8
Kernel: 4.19.0-12-2-amd64
Build commit: ce3b2cb
Build date: Sun Feb 21 10:23:58 UTC 2021
Platform: x86_64-accton_wedge100bf_32x-r0
HwSKU: montara
ASIC: barefoot

Supported testbed topology if it's a new test case?

T0, T1

Documentation

Signed-off-by: Oleksandr Kozodoi <[email protected]>

OleksandrKozodoi · 2020-12-17T09:11:24Z

retest vsimage please

OleksandrKozodoi · 2020-12-17T10:06:02Z

@yxieca @wangxin @lguohan Please review

tests/sub_port_interfaces/sub_ports_helpers.py

tests/sub_port_interfaces/test_sub_port_interfaces.py

OleksandrKozodoi · 2021-01-11T19:05:41Z

@wangxin I am going to add new changes after the merging of #1573

Signed-off-by: Oleksandr Kozodoi <[email protected]>

OleksandrKozodoi · 2021-02-23T14:58:18Z

@yxieca @wangxin @lguohan I've added new changes. Please review.

SavchukRomanLv · 2021-03-01T07:29:13Z

@wangxin @yxieca @lguohan can you take a look? Thank you in advance.

Signed-off-by: Oleksandr Kozodoi <[email protected]>

tests/sub_port_interfaces/conftest.py

tests/sub_port_interfaces/test_sub_port_interfaces.py

wangxin · 2021-03-02T08:50:48Z

tests/sub_port_interfaces/test_sub_port_interfaces.py

+        sub_ports_new[sub_ports.keys()[0]] = sub_ports[sub_ports.keys()[0]]
+        sub_ports_new[sub_ports.keys()[-1]] = sub_ports[sub_ports.keys()[-1]]
+
+        rand_sub_ports = sub_ports.keys()[random.randint(1, len(sub_ports)-1)]


Shall the length of sub_ports (256 in this case) be verified here?

The length of sub_ports is controlled in the block of the define_sub_ports_configuration fixture. The test case uses range of sub-ports from 1 to 256 for port and range of sub-ports from 1 to 96 for lag-port.

wangxin · 2021-03-02T08:52:46Z

tests/sub_port_interfaces/conftest.py

+        # For example: 'PortChannel1.96'
+        if request.param == 'port_in_lag':
+            vlan_ranges_dut = range(1, 97, 24)
+            vlan_ranges_ptf = range(1, 97, 24)


Can you add comments elaborating why 256 sub-ports configuration is generated here?

Linux has the limitation of 15 characters on an interface name and name of LAG port should have prefix 'PortChannel' and suffix '<0-9999>' on SONiC. So max length of sub-port suffix can be 4 characters (such as, PortChannelX.XX, 15 characters). If we use just port, suffix can be 7 charecters (such as, EthernetXXX.XXX, 15 characters).

For test case test_max_numbers_of_sub_ports, is it suppose to create totally 256 number of sub-ports, or just verify that sub-port with suffix like PortChannelX.XX can be created?

I picked define_sub_ports_configuration and tested it from my side on t0. For test case with max_numbers in name, the dicts generated by this fixture are like below:

{ "sub_ports": { "Ethernet4.129": { "ip": "172.16.0.9/30", "neighbor_port": "eth1.129", "neighbor_ip": "172.16.0.10/30" }, "Ethernet8.129": { "ip": "172.16.4.9/30", "neighbor_port": "eth2.129", "neighbor_ip": "172.16.4.10/30" }, "Ethernet8.65": { "ip": "172.16.4.5/30", "neighbor_port": "eth2.65", "neighbor_ip": "172.16.4.6/30" }, "Ethernet4.1": { "ip": "172.16.0.1/30", "neighbor_port": "eth1.1", "neighbor_ip": "172.16.0.2/30" }, "Ethernet4.65": { "ip": "172.16.0.5/30", "neighbor_port": "eth1.65", "neighbor_ip": "172.16.0.6/30" }, "Ethernet4.193": { "ip": "172.16.0.13/30", "nei\r\nghbor_port": "eth1.193", "neighbor_ip": "172.16.0.14/30" }, "Ethernet8.1": { "ip": "172.16.4.1/30", "neighbor_port": "eth2.1", "neighbor_ip": "172.16.4.2/30" }, "Ethernet8.193": { "ip": "172.16.4.13/30", "neighbor_port": "eth2.193", "neighbor_ip": "172.16.4.14/30" } }, "dut_ports": { "1": "Ethernet4", "2": "Ethernet8" }, "ptf_ports": [ "eth1", "eth2" ] }

and

{ "sub_ports": { "PortChannel1.1": { "ip": "172.16.0.1/30", "neighbor_port": "bond1.1", "neighbor_ip": "172.16.0.2/30" }, "PortChannel2.73": { "ip": "172.16.4.\r\n13/30", "neighbor_port": "bond2.73", "neighbor_ip": "172.16.4.14/30" }, "PortChannel2.49": { "ip": "172.16.4.9/30", "neighbor_port": "bond2.49", "neighbor_ip": "172.16.4.10/30" }, "PortChannel1.25": { "ip": "172.\r\n16.0.5/30", "neighbor_port": "bond1.25", "neighbor_ip": "172.16.0.6/30" }, "PortChannel2.25": { "ip": "172.16.4.5/30", "neighbor_port": "bond2.25", "neighbor_ip": "172.16.4.6/30" }, "PortChannel1.49": { "ip": "172.16.0.9/30", "neighbor_port": "bond1.49", "neighbor_ip": "172.16.0.10/30" }, "PortChannel1.73": { "ip": "172.16.0.13/30", "neighbor_port": "bond1.73", "neighbor_ip": "172.16.0.14/30" }, "PortChannel2.1": { "ip": "172.16.4.1/30", "neighbor_port": "bond2.1", "neighbor_ip": "172.16.4.2/30" } }, "dut_ports": { "1": "PortChannel1", "2": "PortChannel2" }, "ptf_ports": { "bond1": "eth1", "bond2": "eth2" } }

The total number of sub-ports is far less than 256. Is this intended?

Test case test_max_numbers_of_sub_ports must create 256 sub-ports for port and 96 sub-ports for lag-port. Thanks for your help in investigating. I am working on this issue.

This issue is fixed by changing steps of range in block (64 to 1), but the test needs too much time (~70 minutes) to run in this case. This behavior due to running of CLI commands for test setup and test teardown on the PTF and DUT. Does it make sense to change the steps of the range for that test case?

Can you try the shell_cmds module? By using this module, you can put all the CLI commands to be executed in a list. Then use this shell_cmds module to run the commands one by one on remote host. It is more efficient than calling the shell module for each CLI command. Can you try it to see how much time is required then?

I've tried the shell_cmds module. By using this module, the test case needs 40 minutes to run.

Do we need to change the sub-ports range in this test case?

I think 40 minutes is still too long. What about this idea:

Add an option for specifying how many sub-ports to be covered. Value range of this option should be 1-256.

Default value of the option can be much lesser to limit the test execution time.
Then we can run this test case by specifying value 256 for the option to have a full coverage when necessary. During daily regression, we can just cover the default range.

I've added --max_numbers_of_sub_ports option for test_max_numbers_of_sub_ports case. Example of the command line how to run this tests:
py.test --testbed=testbed-t0 --inventory=../ansible/lab --testbed_file=../ansible/testbed.csv --host-pattern=testbed-t0 -- module-path=../ansible/library sub_port_interfaces --max_numbers_of_sub_ports=100

Signed-off-by: Oleksandr Kozodoi <[email protected]>

azure-pipelines · 2021-03-11T10:56:17Z

Commenter does not have sufficient privileges for PR 2669 in repo Azure/sonic-mgmt

OleksandrKozodoi · 2021-03-11T17:26:32Z

@wangxin Could you please review?

wangxin · 2021-03-12T02:45:55Z

/azp run

azure-pipelines · 2021-03-12T02:46:06Z

Azure Pipelines successfully started running 1 pipeline(s).

…nux-kernel] advance submodule head (sonic-net#13906) linkmgrd: * 3e7a9df 2023-02-19 | [active-active] Toggle to standby if default route is missing (sonic-net#171) (HEAD -> 202205) [Longxiang Lyu] * 8ab1b2b 2023-02-15 | [active-active] fix issue that interfaces get stuck in `active` if service starts up with link state down (sonic-net#169) [Jing Zhang] * df862ad 2023-02-11 | Fix mux config when gRPC connection is lost (sonic-net#166) [Longxiang Lyu] utilities: * 8aa7930c 2023-02-13 | [portstat CLI] don't print reminder if use json format (sonic-net#2670) (HEAD -> 202205, github/202205) [wenyiz2021] * 4e3bb6fa 2023-02-21 | Add "show fabric reachability" command. (sonic-net#2672) [jfeng-arista] * 3587a94b 2023-02-18 | [202205][dhcp_relay] Remove add field of vlanid to DHCP_RELAY table while adding vlan (sonic-net#2680) [Yaqiang Zhu] * 4f07f7f0 2023-02-10 | Skip saidump for Spine Router as this can take more than 5 sec (sonic-net#2637) (sonic-net#2671) [kenneth-arista] * e61c5ec4 2023-02-10 | [vlan] Refresh dhcpv6_relay config while adding/deleting a vlan (sonic-net#2660) (sonic-net#2669) [Yaqiang Zhu] swss: * 1bbf725 2023-02-14 | [Workaround] EvpnRemoteVnip2pOrch warmboot check failure (sonic-net#2626) (HEAD -> 202205) [jcaiMR] * 380f72b 2023-02-20 | Support for tc-dot1p and tc-dscp qosmap (sonic-net#2559) [Divya Mukundan] * dbf6fcc 2022-11-01 | Added LAG member check on addLagMember() (sonic-net#2464) [Andriy Kokhan] swss-common: * b31391b 2023-02-21 | Prevent sonic-db-cli generate core dump (sonic-net#749) (HEAD -> 202205) [Hua Liu] * 16ff689 2022-12-13 | Support for TC-DOT1p qos map (sonic-net#721) [Divya Mukundan] platform-daemons: * fb92af4 2023-02-09 | [ycabled] add more coverage to ycabled; add minor name change for vendor API CLI return key-values pairs (sonic-net#338) (HEAD -> 202205) [vdahiya12] linux-kernel: * 4e62401 2023-02-09 | Update linux kernel for hw-mgmt V.7.0020.4104 (sonic-net#305) (HEAD -> 202205) [Stephen Sun] Signed-off-by: Ying Xie <[email protected]>

…onic-net#14169) swss: * 2447754 2023-02-23 | Do not allow to add port to .1Q bridge while router port deletion is not completed (sonic-net#2669) (HEAD -> 202205) [Lior Avramov] * 641c65c 2023-03-07 | [bfdorch] add default TOS value for BFD session (sonic-net#2689) (HEAD -> 202205) [Baorong Liu] * 95d3f6d 2023-03-08 | [ACL] Write ACL table/rule creation status into STATE_DB (sonic-net#2662) [bingwang-ms] swss-common: * 1fdeb9e 2023-03-03 | Define ACL_TABLE and ACL_RULE table in STATE_DB (sonic-net#748) (HEAD -> 202205, github/202205) [bingwang-ms] platform-common: * c441bd7 2023-03-07 | [sonic_ssd] Nokia-7215: Fix "show platform ssdhealth" (sonic-net#337) (HEAD -> 202205) [Pavan-Nokia] Signed-off-by: Ying Xie <[email protected]>

Why I did it submodule advance b085b5f - [ci] Fix pipeline error about team5 not found. (Core dump in orchagent when assigning router interface to a vlan with untagged mode sonic-net#2684) (3 hours ago) [Liu Shilong] 4549b4c - Fix issue: there is no retry while creating a RIF which is in removing state ([201811 sub-module] advance sub-modules: utilities, swss, swss-common sonic-net#2679) (3 hours ago) [Junchao-Mellanox] 980a45b - [FDB]Fixing FDB consolidated flush for Remote MACs (pmon to stretch sonic-net#2673) (3 hours ago) [Sudharsan Dhamal Gopalarathnam] c646607 - Do not allow to add port to .1Q bridge while router port deletion is not completed (Update SDK, FW and SAI sonic-net#2669) (3 hours ago) [Lior Avramov] 4a321f0 - [orchagent]: Get bridge port ID from orchagent cache instead of SAI API ([201811 sub module] advance sairedis sub module sonic-net#2657) (3 hours ago) [Lawrence Lee] f4b88f3 - [Dual-ToR] handle 'mux_tunnel_egress_acl' attrib in order to change ACL configuration (drop on ingress/egress) on standby ToR (lm75 doesn't support written alarm to syslog. sonic-net#2646) (3 hours ago) [Andriy Yurkiv] a4f29c1 - [Workaround] EvpnRemoteVnip2pOrch warmboot check failure ([teamd]: wait for swss db flush done before starting teamd container sonic-net#2626) (3 hours ago) [jcaiMR] 53ee0a8 - Support for tc-dot1p and tc-dscp qosmap ([201803] [router-advertiser] Add templated script to wait for pertinent interfaces to be ready before starting radvd sonic-net#2559) (3 hours ago) [Divya Mukundan] b953866 - [dual-tor] add missing SAI attribte in order to create IPNIP tunnel (Config reload/load_minigraph not clearing State DB sonic-net#2503) (3 hours ago) [Andriy Yurkiv] How I did it How to verify it

[Sub-ports] Added new sub-ports test cases

44ed7ec

Signed-off-by: Oleksandr Kozodoi <[email protected]>

yxieca requested a review from a team December 23, 2020 00:00

wangxin reviewed Jan 5, 2021

View reviewed changes

tests/sub_port_interfaces/sub_ports_helpers.py Outdated Show resolved Hide resolved

tests/sub_port_interfaces/test_sub_port_interfaces.py Show resolved Hide resolved

OleksandrKozodoi added 2 commits January 26, 2021 04:52

[Sub-ports] Updated new sub-ports test cases

6b09c33

Signed-off-by: Oleksandr Kozodoi <[email protected]>

Added support of LAG ports and t1 topology

d27cd74

Signed-off-by: Oleksandr Kozodoi <[email protected]>

OleksandrKozodoi requested a review from wangxin February 23, 2021 14:58

Changed approach to set up vlan for test case

fecbcd1

Signed-off-by: Oleksandr Kozodoi <[email protected]>

wangxin reviewed Mar 2, 2021

View reviewed changes

Removed ptfadapter fixture from mtu test case

48d084b

Signed-off-by: Oleksandr Kozodoi <[email protected]>

OleksandrKozodoi requested a review from wangxin March 9, 2021 16:03

Added max_numbers_of_sub_ports options

ace44d1

Signed-off-by: Oleksandr Kozodoi <[email protected]>

wangxin approved these changes Mar 12, 2021

View reviewed changes

wangxin merged commit 38ad99a into sonic-net:master Mar 12, 2021

Conversation

OleksandrKozodoi commented Dec 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

OleksandrKozodoi commented Dec 17, 2020

Uh oh!

OleksandrKozodoi commented Dec 17, 2020

Uh oh!

Uh oh!

Uh oh!

OleksandrKozodoi commented Jan 11, 2021

Uh oh!

OleksandrKozodoi commented Feb 23, 2021

Uh oh!

SavchukRomanLv commented Mar 1, 2021

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

azure-pipelines bot commented Mar 11, 2021

Uh oh!

OleksandrKozodoi commented Mar 11, 2021

Uh oh!

wangxin commented Mar 12, 2021

Uh oh!

azure-pipelines bot commented Mar 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

OleksandrKozodoi commented Dec 15, 2020 •

edited

Loading