pytest for everflow. by abdosi · Pull Request #1435 · sonic-net/sonic-mgmt

abdosi · 2020-03-10T00:33:16Z

Summary:
Adding pytest for everflow.

Type of change

[] Bug fix
[] Testbed and Framework(new/improvement)
[*] Test case(new/improvement)

Approach

How did you do it?

Testcase same as anisble playbook

How did you verify/test it?

py.test -vvv --inventory veos.vtb --host-pattern all --user admin --testbed vms13-5-t1-lag --testbed_file vtestbed.csv --disable_loganalyzer --show-capture stdout --junitxml=tr.xml test_everflow_testbed.py

Result from Broadcom Platform:

test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case1[tor] PASSED [ 2%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case1[spine] PASSED [ 4%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case2[tor] PASSED [ 6%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case2[spine] PASSED [ 8%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case3[tor] PASSED [ 10%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case3[spine] PASSED [ 12%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case4[tor] PASSED [ 14%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case4[spine] PASSED [ 16%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case5[tor] PASSED [ 18%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case5[spine] PASSED [ 20%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case6[tor] PASSED [ 22%]
test_everflow_testbed.py::TestEverflowIngressAclIngressMirror::test_everflow_case6[spine] PASSED [ 25%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case1[tor] SKIPPED [ 27%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case1[spine] SKIPPED [ 29%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case2[tor] SKIPPED [ 31%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case2[spine] SKIPPED [ 33%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case3[tor] SKIPPED [ 35%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case3[spine] SKIPPED [ 37%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case4[tor] SKIPPED [ 39%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case4[spine] SKIPPED [ 41%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case5[tor] SKIPPED [ 43%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case5[spine] SKIPPED [ 45%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case6[tor] SKIPPED [ 47%]
test_everflow_testbed.py::TestEverflowIngressAclEgressMirror::test_everflow_case6[spine] SKIPPED [ 50%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case1[tor] SKIPPED [ 52%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case1[spine] SKIPPED [ 54%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case2[tor] SKIPPED [ 56%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case2[spine] SKIPPED [ 58%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case3[tor] SKIPPED [ 60%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case3[spine] SKIPPED [ 62%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case4[tor] SKIPPED [ 64%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case4[spine] SKIPPED [ 66%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case5[tor] SKIPPED [ 68%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case5[spine] SKIPPED [ 70%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case6[tor] SKIPPED [ 72%]
test_everflow_testbed.py::TestEverflowEgressAclIngressMirror::test_everflow_case6[spine] SKIPPED [ 75%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case1[tor] SKIPPED [ 77%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case1[spine] SKIPPED [ 79%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case2[tor] SKIPPED [ 81%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case2[spine] SKIPPED [ 83%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case3[tor] SKIPPED [ 85%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case3[spine] SKIPPED [ 87%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case4[tor] SKIPPED [ 89%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case4[spine] SKIPPED [ 91%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case5[tor] SKIPPED [ 93%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case5[spine] SKIPPED [ 95%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case6[tor] SKIPPED [ 97%]
test_everflow_testbed.py::TestEverflowEgressAclEgressMirror::test_everflow_case6[spine] SKIPPED [100%]

------------------------- generated xml file: /var/sonicbld/public/tests/tr.xml -------------------------
================================ 12 passed, 36 skipped in 539.47 seconds ================================

Any platform specific information?

N/A

Supported testbed topology if it's a new test case?

T1 topology

tests/test_everflow_testbed.py

daall

LGTM as a whole, I have some small cosmetic suggestions but I don't see any blocking issues here. Will wait for you to reply and then approve. :)

tests/test_everflow_testbed.py

lguohan · 2020-03-16T22:52:27Z

can you check on the mlnx platform?

lguohan · 2020-03-17T05:56:51Z

can you use pytest_runner to replace the original yml base test?

abdosi · 2020-03-18T01:49:29Z

can you check on the mlnx platform?

@lguohan verified on 201811 image for Ingress Mirror/Ingress ACL.
py.test -vvv --inventory veos.vtb --host-pattern all --user admin --testbed vms11-2-t1 --testbed_file vtestbed.csv --disable_loganalyzer --show-capture stdout --junitxml=tr.xml test_everflow_testbed.py

abdosi · 2020-03-18T06:29:15Z

can you use pytest_runner to replace the original yml base test?

Updated to use pytest_runner for everflow

combination of ingress/egress acl with ingress/egress mirror and port as tor/spine. Made change to add dedicated route for regular traffic. Addresed Review Comment. Also make everflow to use pytest runner. Added more comment for partial_ptf_runner.

- swss: * 459d09b 2021-02-22 | [acl] Enable VLAN ID qualifier for ACL rules (sonic-net#1648) (HEAD -> 202012) [Danny Allen] * 60859b1 2021-02-08 | [buffermgr] Support maximum port headroom checking (sonic-net#1607) [Stephen Sun] * 3161cbb 2021-02-17 | Add SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS counter, create new FlexCounter group (sonic-net#1600) [Andriy Yurkiv] * 256ed9c 2021-02-08 | Support shared headroom pool on top of dynamic buffer calculation (sonic-net#1581) [Stephen Sun] - utilities: * 30d7069 2021-02-16 | [decode-syseeprom] Refactor to utilize sonic-platform package (sonic-net#1435) (HEAD -> 202012, tag: foo) [Joe LeVeque] * 891fef4 2021-02-16 | [psuutil] Refactor to utilize sonic-platform package (sonic-net#1434) [Joe LeVeque] * 8c5e505 2021-02-17 | [sfputil] Refactor to utilize sonic-platform package (sonic-net#1421) [Joe LeVeque] * ca5dd2c 2021-02-17 | [sfpshow] Cleanup (sonic-net#1405) [Joe LeVeque] * bf489ea 2021-02-17 | Add new cli for SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS counter in counterpoll utility (sonic-net#1355) [Andriy Yurkiv] * 25feed3 2021-02-08 | Support shared headroom pool on top of dynamic buffer calculation (sonic-net#1348) [Stephen Sun] * aaa323a 2021-02-02 | [vrf]: Fix freezing during interface binding (sonic-net#1325) [maksymbelei95] * cc0bb6e 2021-01-27 | [show] fix "show interfaces breakout" command (sonic-net#1198) [Dmytro Shevchuk] Signed-off-by: Ying Xie <ying.xie@microsoft.com>

abdosi added Enhancement InProgress labels Mar 10, 2020

abdosi assigned daall, lguohan and yxieca Mar 10, 2020

abdosi force-pushed the everflow_pytest branch from 3a4ee8c to bebd8c9 Compare March 10, 2020 02:15

yxieca reviewed Mar 13, 2020

View reviewed changes

tests/test_everflow_testbed.py Outdated Show resolved Hide resolved

yxieca reviewed Mar 13, 2020

View reviewed changes

tests/test_everflow_testbed.py Outdated Show resolved Hide resolved

abdosi force-pushed the everflow_pytest branch from bebd8c9 to 0b8edb1 Compare March 13, 2020 21:36

abdosi removed the InProgress label Mar 13, 2020

abdosi force-pushed the everflow_pytest branch from 0b8edb1 to 0411558 Compare March 14, 2020 01:53

lguohan force-pushed the master branch from eca7906 to 4d15a53 Compare March 15, 2020 09:25

yxieca approved these changes Mar 16, 2020

View reviewed changes

daall reviewed Mar 16, 2020

View reviewed changes

abdosi force-pushed the everflow_pytest branch from 0411558 to cd0fedc Compare March 18, 2020 06:28

abdosi force-pushed the everflow_pytest branch from cd0fedc to 82dc1f1 Compare March 18, 2020 15:34

daall approved these changes Mar 20, 2020

View reviewed changes

abdosi merged commit 3f24345 into sonic-net:master Mar 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pytest for everflow. #1435

pytest for everflow. #1435
abdosi merged 1 commit intosonic-net:masterfrom
abdosi:everflow_pytest

abdosi commented Mar 10, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

daall left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lguohan commented Mar 16, 2020

Uh oh!

lguohan commented Mar 17, 2020

Uh oh!

abdosi commented Mar 18, 2020

Uh oh!

abdosi commented Mar 18, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

abdosi commented Mar 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of change

Approach

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Uh oh!

Uh oh!

Uh oh!

daall left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lguohan commented Mar 16, 2020

Uh oh!

lguohan commented Mar 17, 2020

Uh oh!

abdosi commented Mar 18, 2020

Uh oh!

abdosi commented Mar 18, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

abdosi commented Mar 10, 2020 •

edited

Loading