[tests/mclag] Submit mclag testcases according to mclag test plan by freeman386 · Pull Request #1239 · sonic-net/sonic-mgmt

freeman386 · 2019-11-27T08:00:39Z

Description of PR

Summary:
Submit MCLAG testcases according to mclag test plan [Azure/SONiC/pull/422].

Dependencies:

MCLAG feature for SONIC [Azure/sonic-buildimage/pull/2514]
MCLAG topo deployment [Azure/sonic-mgmt/pull/1251]
Need install teamd on PTF

Type of change

[] Bug fix
[] Testbed and Framework(new/improvement)
Test case(new/improvement)

Approach

How did you do it?

mclag test on test_scenario L2

root@sonic-mgmt:/var/root/sonic-mgmt-main/tests# pytest -vvv --disable_loganalyzer --inventory veos --host-pattern all --user admin --testbed vms-t0-mclag-16 --testbed_file testbed.csv --show-capture=stdout test_mclag.py                                    
====================================================================== test session starts =======================================================================
platform linux2 -- Python 2.7.12, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python
cachedir: .pytest_cache
ansible: 2.0.0.2
rootdir: /var/root/sonic-mgmt-main/tests, inifile: pytest.ini
plugins: repeat-0.8.0, ansible-2.0.2
collected 56 items                                                                                                                                               

test_mclag.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link PASSED                                                                               [  1%]
test_mclag.py::TestCase1_VerifyMclagStatus::test_check_teamd_system_id PASSED                                                                              [  3%]
test_mclag.py::TestCase1_VerifyMclagStatus::test_traffic_between_servers PASSED                                                                            [  5%]
test_mclag.py::TestCase1_VerifyMclagStatus::test_syncheck[syncheck0] PASSED                                                                                [  7%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_traffic_between_servers PASSED                                                                  [  8%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_down PASSED                                                                   [ 10%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_up PASSED                                                                     [ 12%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_syncheck[syncheck0] PASSED                                                                      [ 14%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_traffic_between_servers PASSED                                                                         [ 16%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_peer_link_interface_down PASSED                                                                        [ 17%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_syncheck_on_mclag_interface[syncheck0] PASSED                                                          [ 19%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_peer_link_interface_up PASSED                                                                          [ 21%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_syncheck[syncheck0] PASSED                                                                             [ 23%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_traffic_between_servers PASSED                                                                    [ 25%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_keepalive_link_down PASSED                                                                        [ 26%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_keepalive_link_up PASSED                                                                          [ 28%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_syncheck[syncheck0] PASSED                                                                        [ 30%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_traffic_between_servers PASSED                                                                [ 32%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_peer_keepalive_link_down PASSED                                                               [ 33%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_peer_keepalive_link_up PASSED                                                                 [ 35%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_syncheck[syncheck0] PASSED                                                                    [ 37%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_traffic_between_servers PASSED                                                                        [ 39%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_active_down PASSED                                                                                    [ 41%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_active_up PASSED                                                                                      [ 42%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_syncheck[syncheck0] PASSED                                                                            [ 44%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_traffic_between_servers PASSED                                                                       [ 46%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_standby_down PASSED                                                                                  [ 48%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_standby_up PASSED                                                                                    [ 50%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_syncheck[syncheck0] PASSED                                                                           [ 51%]
test_mclag.py::TestCase8_ActiveDevWarmreboot::test_traffic_between_servers PASSED                                                                          [ 53%]
test_mclag.py::TestCase8_ActiveDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                        [ 55%]
test_mclag.py::TestCase8_ActiveDevWarmreboot::test_syncheck_after_warmreboot[syncheck0] PASSED                                                             [ 57%]
test_mclag.py::TestCase9_StandbyDevWarmreboot::test_traffic_between_servers PASSED                                                                         [ 58%]
test_mclag.py::TestCase9_StandbyDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                       [ 60%]
test_mclag.py::TestCase9_StandbyDevWarmreboot::test_syncheck_after_warmreboot[syncheck0] PASSED                                                            [ 62%]
test_mclag.py::TestCase10_MacFlapping::test_traffic_before_mac_flapping PASSED                                                                             [ 64%]
test_mclag.py::TestCase10_MacFlapping::test_syncheck_before_mac_flapping[syncheck0] PASSED                                                                 [ 66%]
test_mclag.py::TestCase10_MacFlapping::test_mac_flapping PASSED                                                                                            [ 67%]
test_mclag.py::TestCase11_MacSyncAndAge::test_traffic_between_servers PASSED                                                                               [ 69%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_mclag_member_port_down PASSED                                                           [ 71%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_mclag_member_port_up PASSED                                                             [ 73%]
test_mclag.py::TestCase11_MacSyncAndAge::test_traffic_between_servers_after_mclag_member_port_up PASSED                                                    [ 75%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_traffic_between_servers PASSED                                                          [ 76%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_before_age PASSED                                                                             [ 78%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mac_aging_on_peer PASSED                                                                                     [ 80%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_aging PASSED                                                                            [ 82%]
test_mclag.py::TestCase11_MacSyncAndAge::test_relearn_after_mac_age PASSED                                                                                 [ 83%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_relearn PASSED                                                                          [ 85%]
test_mclag.py::TestCase12_ICCP_CSM::test_active_restart PASSED                                                                                             [ 87%]
test_mclag.py::TestCase12_ICCP_CSM::test_standby_restart PASSED                                                                                            [ 89%]
test_mclag.py::TestCase13_Scaling::test_traffic_between_servers PASSED                                                                                     [ 91%]
test_mclag.py::TestCase13_Scaling::test_mac_arp_sync PASSED                                                                                                [ 92%]
test_mclag.py::TestCase14_CornerTest::test_stop_teamd_on_active PASSED                                                                                     [ 94%]
test_mclag.py::TestCase14_CornerTest::test_start_teamd_on_active PASSED                                                                                    [ 96%]
test_mclag.py::TestCase14_CornerTest::test_stop_teamd_on_standby PASSED                                                                                    [ 98%]
test_mclag.py::TestCase14_CornerTest::test_start_teamd_on_standby PASSED                                                                                   [100%]

======================================================================== warnings summary ========================================================================
test_mclag.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link
  /usr/local/lib/python2.7/dist-packages/pytest_ansible/module_dispatcher/v2.py:70: UserWarning: provided hosts list is empty, only localhost is available
    warnings.warn("provided hosts list is empty, only localhost is available")

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================ 56 passed, 1 warnings in 5482.20 seconds ============================================================

mclag test on test_scenario L3

root@sonic-mgmt:/var/root/sonic-mgmt-main/tests# pytest -vvv --disable_loganalyzer --inventory veos --host-pattern all --user admin --testbed vms-t0-mclag-16 --testbed_file testbed.csv --show-capture=stdout test_mclag_l3.py 
====================================================================== test session starts =======================================================================
platform linux2 -- Python 2.7.12, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python
cachedir: .pytest_cache
ansible: 2.0.0.2
rootdir: /var/root/sonic-mgmt-main/tests, inifile: pytest.ini
plugins: repeat-0.8.0, ansible-2.0.2
collected 24 items                                                                                                                                               

test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link PASSED                                                                            [  4%]
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_check_teamd_system_id PASSED                                                                           [  8%]
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_traffic_between_servers PASSED                                                                         [ 12%]
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_syncheck PASSED                                                                                        [ 16%]
test_mclag_l3.py::TestCase2_MclagMemberPortStatusChange::test_traffic_between_servers PASSED                                                               [ 20%]
test_mclag_l3.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_down PASSED                                                                [ 25%]
test_mclag_l3.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_up PASSED                                                                  [ 29%]
test_mclag_l3.py::TestCase3_KeepaliveLinkStatusChange::test_traffic_between_servers PASSED                                                                 [ 33%]
test_mclag_l3.py::TestCase3_KeepaliveLinkStatusChange::test_keepalive_link_down PASSED                                                                     [ 37%]
test_mclag_l3.py::TestCase3_KeepaliveLinkStatusChange::test_keepalive_link_up PASSED                                                                       [ 41%]
test_mclag_l3.py::TestCase4_ActiveDevStatusChange::test_traffic_between_servers PASSED                                                                     [ 45%]
test_mclag_l3.py::TestCase4_ActiveDevStatusChange::test_active_down PASSED                                                                                 [ 50%]
test_mclag_l3.py::TestCase4_ActiveDevStatusChange::test_active_up PASSED                                                                                   [ 54%]
test_mclag_l3.py::TestCase5_StandbyDevStatusChange::test_traffic_between_servers PASSED                                                                    [ 58%]
test_mclag_l3.py::TestCase5_StandbyDevStatusChange::test_standby_down PASSED                                                                               [ 62%]
test_mclag_l3.py::TestCase5_StandbyDevStatusChange::test_standby_up PASSED                                                                                 [ 66%]
test_mclag_l3.py::TestCase6_ActiveDevWarmreboot::test_traffic_between_servers PASSED                                                                       [ 70%]
test_mclag_l3.py::TestCase6_ActiveDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                     [ 75%]
test_mclag_l3.py::TestCase6_ActiveDevWarmreboot::test_syncheck PASSED                                                                                      [ 79%]
test_mclag_l3.py::TestCase7_StandbyDevWarmreboot::test_traffic_between_servers PASSED                                                                      [ 83%]
test_mclag_l3.py::TestCase7_StandbyDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                    [ 87%]
test_mclag_l3.py::TestCase7_StandbyDevWarmreboot::test_syncheck PASSED                                                                                     [ 91%]
test_mclag_l3.py::TestCase8_Scaling::test_traffic_between_servers PASSED                                                                                   [ 95%]
test_mclag_l3.py::TestCase8_Scaling::test_syncheck PASSED                                                                                                  [100%]

======================================================================== warnings summary ========================================================================
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link
  /usr/local/lib/python2.7/dist-packages/pytest_ansible/module_dispatcher/v2.py:70: UserWarning: provided hosts list is empty, only localhost is available
    warnings.warn("provided hosts list is empty, only localhost is available")

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================ 24 passed, 1 warnings in 2234.17 seconds ============================================================

Supported testbed topology if it's a new test case?

According test plan, we build mclag topo based on topo-t0, supported for topo_t0-mclag(which has 32 dut ports) and topo_t0-mclag-16(which has 16 dut ports).

Documentation

https://github.com/shine4chen/SONiC/blob/mclag-test-case/doc/mclag_hld/MCLAG-ansible-test-plan.md

chenkelly · 2020-07-01T07:49:57Z

Hi @freeman386 and @William-zx
Could you guide us how to install teamd on PTF and run it successfully?
We try to install it (https://github.com/jpirko/libteam), but get error when we execute.
root@9a5d2e8482f3:/home# teamd This program is not intended to be run as root. Failed to create team device. teamd_init() failed. Failed: Operation not permitted
The source code of teamd does not allow the execution by root.
We also try remove code checking for root limitation, but get other issues.

Has you experienced this?
Please advise us your suggestion.
Thanks very much.

William-zx · 2020-07-01T08:40:21Z

@chenkelly
"teamd This program is not intended to be run as root." This prompt will also appear in our environment, but the teamd works normally.
Two ways to install teamd on PTF:

Copy all teamd-related DEBs to the PTF and install them. Then use "docker commit" command to generate a new image.
Modify the makefile of ptf in sonic-buildimage. Then bulid new ptf image. For example:

dockers/docker-ptf/Dockerfile.j2 
+        libdbus-1-3         \
+        libdaemon0          \
+        libjansson4         \

platform/nephos/docker-ptf-nephos.mk 
-$(DOCKER_PTF_NEPHOS)_DEPENDS += $(PYTHON_SAITHRIFT_NEPHOS)
+$(DOCKER_PTF_NEPHOS)_DEPENDS += $(PYTHON_SAITHRIFT) $(LIBTEAM) $(LIBTEAMDCT) $(LIBTEAM_UTILS) $(LIBNL3) $(LIBNL_GENL3) $(LIBNL_ROUTE3) $(LIBNL_NF3) $(LIBNL_CLI)

make configure PLATFORM=nephos
make target/docker-ptf-nephos.gz

chenkelly · 2020-07-02T08:50:09Z

Hi @William-zx
Thanks. We use method 1 to generate PTF image which supports temad.
After we add t0-mclag topology on sonic-mgmt docker, we using "dmesg" command to check server status, but find that some error messages as follows.
[84977.213102] eth56: renamed from ens2f0.2457 [84977.247906] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247928] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247948] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247957] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.602181] eth57: renamed from ens2f0.2458 [84980.628172] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628190] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628211] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628230] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28
Has you experienced this? Our testbed server OS is ubuntu 16.04. Is it the appropriate version? Thanks.

William-zx · 2020-07-06T03:27:02Z

Hi @William-zx
Thanks. We use method 1 to generate PTF image which supports temad.
After we add t0-mclag topology on sonic-mgmt docker, we using "dmesg" command to check server status, but find that some error messages as follows.
[84977.213102] eth56: renamed from ens2f0.2457 [84977.247906] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247928] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247948] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247957] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.602181] eth57: renamed from ens2f0.2458 [84980.628172] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628190] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628211] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628230] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28
Has you experienced this? Our testbed server OS is ubuntu 16.04. Is it the appropriate version? Thanks.

@chenkelly
We do not experienced this, my testbed server OS is ubuntu 16.04.2.
You mean that you can add-topo T0/T1/T1-lag/ PTF32 successfully, but add-topo t0-mclag failed?
We just only added a few ports, not modified the common process of add-topo step.

chenkelly · 2020-07-06T03:50:53Z

Hi @William-zx
We can add t0-mclag topology and deploy successfully, but see these message on testbed server.
We are not sure whether test_mclag.py is failed due to these message or not.
After running test_mclag.py, it hang on test case test_mclag.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link and then we cannot remove topology successfully. We are checking configuration during testing.
If you have any suggestions, please tell us. Thanks very much
`[77845.001485] ens2f0.2456: renamed from eth55

[77846.329985] PortChannel24: Port device ens2f0.2456 removed

[77847.495120] ens2f0.2457: renamed from eth56

[77850.126644] ens2f0.2458: renamed from eth57

[77854.348950] device ens2f0.2427 left promiscuous mode

[77856.021577] device ens2f0.2459 left promiscuous mode

[77856.865984] device br-vms2-4-ic-1 left promiscuous mode

[77858.645834] device ens2f0.2428 left promiscuous mode

[77860.421388] device ens2f0.2460 left promiscuous mode

[77861.432696] device br-vms2-4-ic-2 left promiscuous mode

[77873.724953] unregister_netdevice: waiting for ens2f0.2401 to become free. Usage count = 2

[77883.901366] unregister_netdevice: waiting for ens2f0.2401 to become free. Usage count = 2`

chenkelly · 2020-07-14T04:26:03Z

Hi @William-zx
We found root case. On our DUT, Ethernet index is not continuous, and will select incorrect Ethernet port for VLAN member assignment.
https://github.com/freeman386/sonic-mgmt/blob/master/tests/mclag/mclag_configuration.j2#L78
We will consider to use port_alias_map instead of Ethernet index.
Thanks very much.

…tag instead of serial number (sonic-net#17440) (sonic-net#17493) To modify EEPROM API serial_number_str to return service tag instead of serial number in Dell S6100. Ref PR: sonic-net#1239 How I did it Update EEPROM API serial_number_str to return service tag instead of serial number. How to verify it Verify decode-syseeprom -s returns service tag in Dell S6100. Co-authored-by: Arun Saravanan Balachandran <[email protected]>

Revert "[portsorch] Enable port-level buffer drop counters (sonic-net#1237)" (sonic-net#1308) Add more log message, fix test code (sonic-net#1239)

…tag instead of serial number (sonic-net#17440) To modify EEPROM API serial_number_str to return service tag instead of serial number in Dell S6100. Ref PR: sonic-net#1239 How I did it Update EEPROM API serial_number_str to return service tag instead of serial number. How to verify it Verify decode-syseeprom -s returns service tag in Dell S6100.

Felix.Wu added 3 commits November 27, 2019 15:44

link arp_responder files to ansible dirs to remove duplicate files

7090f37

modify path to be called in pytest compatibilitilly

e45d2a3

add mclag test scripts

ca277bd

freeman386 changed the title ~~[tests/mclag] Submit mclag testcases according to mclagtest plan~~ [tests/mclag] Submit mclag testcases according to mclag test plan Nov 27, 2019

William-zx mentioned this pull request Dec 2, 2019

[ansible/MCLAG]: add t0-mclag and t0-mclag-16 typologies support #1251

Open

1 task

freeman386 marked this pull request as ready for review December 2, 2019 08:32

lguohan added the Design Review Needed label Apr 25, 2020

yxieca force-pushed the master branch from 885053e to 8f9cb56 Compare October 31, 2020 02:57

freeman386 requested a review from a team as a code owner March 2, 2022 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[tests/mclag] Submit mclag testcases according to mclag test plan#1239

[tests/mclag] Submit mclag testcases according to mclag test plan#1239
freeman386 wants to merge 3 commits intosonic-net:masterfrom
freeman386:master

freeman386 commented Nov 27, 2019 •

edited

Loading

Uh oh!

chenkelly commented Jul 1, 2020 •

edited

Loading

Uh oh!

William-zx commented Jul 1, 2020 •

edited

Loading

Uh oh!

chenkelly commented Jul 2, 2020

Uh oh!

William-zx commented Jul 6, 2020

Uh oh!

chenkelly commented Jul 6, 2020 •

edited

Loading

Uh oh!

chenkelly commented Jul 14, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

freeman386 commented Nov 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Approach

How did you do it?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

chenkelly commented Jul 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

William-zx commented Jul 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chenkelly commented Jul 2, 2020

Uh oh!

William-zx commented Jul 6, 2020

Uh oh!

chenkelly commented Jul 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chenkelly commented Jul 14, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

freeman386 commented Nov 27, 2019 •

edited

Loading

chenkelly commented Jul 1, 2020 •

edited

Loading

William-zx commented Jul 1, 2020 •

edited

Loading

chenkelly commented Jul 6, 2020 •

edited

Loading