[pytest]: Fail WR ARP test earlier with empty DIP

[theasianpianist]

• Add assertion to arp/test_wr_arp.py to catch an empty DIP returned
  from DUT host

Signed-off-by: Lawrence Lee lawlee@microsoft.com

Description of PR

The WR ARP test case does not fail with a clear/useful error message if it receives an empty DIP from the DUT host.
Summary:
Fixes #1985 (issue)

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Approach

What is the motivation for this PR?

Provide clearer failure messages for the WR ARP test case

How did you do it?

Add an assertion to check the length of the DIP string

How did you verify/test it?

1. Follow steps in [pytest]: WR ARP test fails with unclear message when DIP is empty #1985 to reproduce the error. The test should still fail, but now with the following error message:

>       pytest_assert(len(result['stdout'].strip()) > 0, 'Empty DIP returned')
E       Failed: Empty DIP returned
duthost    = <tests.common.devices.SonicHost object at 
ptfhost    = <tests.common.devices.PTFHost object at 
result     = {'stderr_lines': [], u'cmd': u"ip route show type unicast |\n            sed -...chdir': None, u'stdin_add_newline': True u'stdin': None}}, 'stdout_lines': []}                                                                
self       = <tests.arp.test_wr_arp.TestWrArp instance at 0x7fc7c5827098>
arp/test_wr_arp.py:102: Failed                                                                                                                                                 

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

You should be able to return your DUT to a normal state by restarting the exabgp* processes on the PTF host and running the test_announce_routes.py test case.

[tahmed-dev]

Please remove the redundant assert above.

[theasianpianist]

I'm not sure that this is redundant, it looks like it's checking for the command failing outright rather than just not returning anything. Is it possible for stdout to be contain content when stderr_lines also contains content?

[tahmed-dev]

I suspect the return code will be zero in this case. If it is, the module runner will throw and so the assert is redundant in this case.

[theasianpianist]

Ok sounds good, will fix.

[mithun2498]

Hi @theasianpianist
I tried running this case in T0 topology with a physical DUT and I am facing the following issue. Kindly help me resolve this issue.

Error Log:
result = duthost.shell(
cmd=r'''ip route show type unicast |
sed -e '/proto 186|proto zebra|proto bgp/!d' -e '/default/d' -ne '/0//p' |
head -n 1 |
sed -ne 's/0/.*$/1/p'
'''
)

  pytest_assert(len(result['stdout'].strip()) > 0, 'Empty DIP returned')

E Failed: Empty DIP returned

duthost =
ptfhost = <tests.common.devices.ptf.PTFHost object at 0x7fdeff0173d0>
result = {'stderr_lines': [], u'cmd': u"ip route show type unicast |\n sed -...tdin_add_newline': True, u'stdin': None}}, 'stdout_lines': [], 'failed': False}
self = <tests.arp.test_wr_arp.TestWrArp instance at 0x7fdeff0ba1e0>
tbinfo = {'auto_recover': 'True', 'comment': 'Tests Broadcom AS7716 vms', 'conf-name': 'vms-sn2700-t0', 'duts': ['dev-msn2700-01'], ...}

arp/test_wr_arp.py:106: Failed

• generated xml file: /data/sonic-mgmt/tests/logs/24-04-2023/test_wr_arp/tr.xml -
  ---------------------------- live log sessionfinish ----------------------------
  15:12:39 init.pytest_terminal_summary L0064 INFO | Can not get Allure report URL. Please check logs
  =========================== short test summary info ============================
  ERROR arp/test_wr_arp.py::TestWrArp::testWrArp - Failed: Empty DIP returned
  ERROR arp/test_wr_arp.py::TestWrArp::testWrArpAdvance - Failed: Empty DIP ret...
  ========================== 2 error in 344.01 seconds ===========================

Version Details:
SONiC Software Version: SONiC.202211.269499-59c7d39ef
SONiC OS Version: 11
Distribution: Debian 11.6
Kernel: 5.10.0-18-2-amd64
Build commit: 59c7d39
Build date: Tue May 9 17:58:15 UTC 2023
Built by: AzDevOps@vmss-soni00118K
Platform: x86_64-accton_as7716_32x-r0
HwSKU: Accton-AS7716-32X
ASIC: broadcom
ASIC Count: 1