[pytest] Convert Ansible QoS SAI test cases

[tahmed-dev]

Description of PR

This PR converts Ansible QoS SAI test cases to pytest framework. The
experimental test cases under buff_wm.yml is not part of this conversion.

signed-off-by: Tamer Ahmed tamer.ahmed@microsoft.com

Summary:

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Approach

How did you do it?

New code that ports Ansible QoS SAI test

How did you verify/test it?

mgmt-repo/tests$ pytest qos/test_qos_sai.py --testbed=vms12-t0-s6000-1 --inventory=../ansible/str --testbed_file=../ansible/testbed.csv --host-pattern=str-s6000-acs-14 --module-path=../ansible/library --disable_loganalyzer --skip_sanity
================================================================================================================================================== test session starts ===================================================================================================================================================
platform linux2 -- Python 2.7.12, pytest-4.6.9, py-1.8.1, pluggy-0.13.1
ansible: 2.8.7
rootdir: /var/host-acs-mgmt-repo/tests, inifile: pytest.ini
plugins: ansible-2.2.2
collected 15 items                                                                                                                                                                                                                                                                                                       

qos/test_qos_sai.py ....s........s.                                                                                                                                                                                                                                                                                [100%]

================================================================================================================================== 13 passed, 2 skipped, 1 warnings in 1277.13 seconds ===================================================================================================================================

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[tahmed-dev]

@wangxin here.

[tahmed-dev]

@wangxin args grouping

[lgtm-com]

This pull request introduces 1 alert when merging 8a95e1c9897b554cc504e6b9e524fdf23809754f into eda7bf5 - view on LGTM.com

new alerts:

• 1 for Unused import

[yxieca]

Tamer, did you move some files with 'mv' instead of 'git mv'?

"git mv" will show files as 'renamed without change' like the last few files in this PR.

[tahmed-dev]

Tamer, did you move some files with 'mv' instead of 'git mv'?

"git mv" will show files as 'renamed without change' like the last few files in this PR.

Ying, I did. I also added symlink in the same commit. However some files are showing history ported (those inside the dir saitest) and others do not show the history. I will split into two commits and most probably will not squash commits when checking in.

The history could be seen here:
https://github.com/tahmed-dev/sonic-mgmt/blame/taahme/convert-qos-sai-tc/tests/saitests/sai_qos_tests.py#L11

[tahmed-dev]

Tamer, did you move some files with 'mv' instead of 'git mv'?
"git mv" will show files as 'renamed without change' like the last few files in this PR.

Ying, I did. I also added symlink in the same commit. However some files are showing history ported (those inside the dir saitest) and others do not show the history. I will split into two commits and most probably will not squash commits when checking in.

The history could be seen here:
https://github.com/tahmed-dev/sonic-mgmt/blame/taahme/convert-qos-sai-tc/tests/saitests/sai_qos_tests.py#L11

I did create different commit for file move and another one for symlinks. The history could be see here, here, and here

However the diff is not showing them are removed. This is because of present of symlinks in the commit set. I think if we squash this one we might lose the history. I am thinking of not squashing, @yxieca what do you think?

[yvolynets-mlnx]

From which folder do you expect to run this test cases? I'm asking because looks like if to use "pytest_addoption" hook not inside main conftest file and in the same time run tests from "sonic-mgmt/tests" directory, pytest will raise error that specified options are not known like you define above.
For example if to pass to the console "--qos_dst_ports" option and run tests from "sonic-mgmt/tests" folder.

@tahmed-dev @wangxin What do you think?

[tahmed-dev]

I am expecting the test case to run from tests dir. I did not see an issue with passing argument when addoption is called in sub-conftest files. https://docs.pytest.org/en/2.7.3/plugins.html
Did you have issues running the test? I just ran it with no problem using

pytest qos/test_qos_sai.py --testbed=vms12-t0-s6000-1 --inventory=../ansible/str --testbed_file=../ansible/testbed.csv --host-pattern=str-s6000-acs-14 --module-path=../ansible/library --disable_loganalyzer --skip_sanity --qos_dst_ports=0,2,3 --qos_src_ports=1
================================================================================================================================================== test session starts ===================================================================================================================================================
platform linux2 -- Python 2.7.12, pytest-4.6.9, py-1.8.1, pluggy-0.13.1
ansible: 2.8.7
rootdir: /var/host-acs-mgmt-repo/tests, inifile: pytest.ini
plugins: ansible-2.2.2
collected 1 item                                                                                                                                                                                                                                                                                                         

qos/test_qos_sai.py 

[tahmed-dev]

@yvolynets-mlnx There was an issue of reading list from the command line that I fixed in a recent commit. Please try it out and let me know if you hit any issues.