Vnet vxlan pytest conversion by theasianpianist · Pull Request #2113 · sonic-net/sonic-mgmt

theasianpianist · 2020-08-18T02:55:38Z

Description of PR

Summary:
Fixes # (issue)

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Approach

What is the motivation for this PR?

Migrate VNET VxLAN test to pytest framework

How did you do it?

Create test module for VNET VxLAN
Copy necessary Jinja templates to pytest-specific template directory

How did you verify/test it?

Run pytest and check that all cases (Disabled, Enabled, Cleanup) succeed
Run with scaled options (num_vnet, num_routes, num_endpoints) and check that the given values are reflected in the generated config files
Run with --skip_cleanup and verify that cleanup step is skipped
Run with --ipv6_vxlan_test and verify correct values in generated config files

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Run this test with:

./run_tests.sh -i <inventory file> -d <DUT name> -n <testbed name> -u -c vxlan/test_vnet_vxlan.py

Use extra CLI parameters as following:

./run_tests.sh ...(same as above)... -e "--param_with_value=<value> --param_without_value"

Scaled VNET test:

--num_vnet=<x>
--num_routes=<y>
--num_endpoints=<z>

Other:

--skip_cleanup (don't remove VNET info after test ends)

* Create pytest specific versions of VNET configuration templates * Add VNET VxLAN pytest module Signed-off-by: Lawrence Lee <[email protected]>

* Fix template formatting * Correct config file location for PTF host Signed-off-by: Lawrence Lee <[email protected]>

* Control whitespace in rendered templates to preserve formatting Signed-off-by: Lawrence Lee <[email protected]>

Signed-off-by: Lawrence Lee <[email protected]>

* Allow multiple modules to access this fixture Signed-off-by: Lawrence Lee <[email protected]>

Signed-off-by: Lawrence Lee <[email protected]>

daall

See comments. I did not look super closely at the j2 templates, please check with @prsunny.

tests/conftest.py

tests/vxlan/vnet_config.py

tests/vxlan/vnet_utils.py

tests/vxlan/test_vnet_vxlan.py

prsunny · 2020-08-20T15:55:55Z

tests/vxlan/vnet_utils.py

+    for vnet in vc.VNET_CONFIG['vnet_id_list']:
+        duthost.shell("docker exec -i database redis-cli -n 4 del \"VNET|{}\"".format(vnet))
+
+def cleanup_vxlan_tunnels(duthost):


We would also want to do cleanup of Vlans. This can be ported once ansible fix is available

tests/vxlan/vnet_utils.py

tests/vxlan/test_vnet_vxlan.py

prsunny · 2020-08-20T16:02:11Z

@volodymyrsamotiy , please review

* Move parser addoption method to test specific conftest.py * Bundle parameters for scaled vnet test into dictionary * Bundle general vnet test parameters into dictionary Signed-off-by: Lawrence Lee <[email protected]>

* Change all imports to match * Create vnet_config fixture for value that was previously in constants file Signed-off-by: Lawrence Lee <[email protected]>

tests/conftest.py

tests/templates/vnet_config.j2

tests/vxlan/test_vnet_vxlan.py

* Return instead of yield in setup function * Make variable name clearer * Add pytest topology marker Signed-off-by: Lawrence Lee <[email protected]>

Signed-off-by: Lawrence Lee <[email protected]>

lgtm-com · 2020-08-21T18:41:19Z

This pull request introduces 3 alerts when merging 0bf7d59 into dd43ab3 - view on LGTM.com

new alerts:

2 for 'import *' may pollute namespace
1 for Unused import

Signed-off-by: Lawrence Lee <[email protected]>

tests/vxlan/__init__.py

tests/vxlan/test_vnet_vxlan.py

tests/vxlan/vnet_utils.py

Co-authored-by: Danny Allen <[email protected]>

Signed-off-by: Lawrence Lee <[email protected]>

…t/sonic-mgmt into vnet-vxlan-pytest

Signed-off-by: Lawrence Lee <[email protected]>

theasianpianist · 2020-08-24T23:10:28Z

retest vsimage please

Signed-off-by: Lawrence Lee <[email protected]>

prsunny · 2020-08-27T04:08:54Z

@volodymyrsamotiy for viz

daall

Two really small suggestions, nothing blocking. Be sure to double-check with @prsunny and @volodymyrsamotiy before merging this. :)

tests/vxlan/__init__.py

tests/vxlan/vnet_utils.py

* Fix typo * Update `safe_open_template` method docstring Signed-off-by: Lawrence Lee <[email protected]>

tests/vxlan/vnet_utils.py

theasianpianist added 4 commits August 4, 2020 19:11

[WIP][pytest]: Convert VNET VxLAN test to pytest

789d220

* Create pytest specific versions of VNET configuration templates * Add VNET VxLAN pytest module Signed-off-by: Lawrence Lee <[email protected]>

[pytest]: Fix VNET VxLAN test errors

927b423

* Fix template formatting * Correct config file location for PTF host Signed-off-by: Lawrence Lee <[email protected]>

[pytest]: Improve rendered template readability

c6fc56b

* Control whitespace in rendered templates to preserve formatting Signed-off-by: Lawrence Lee <[email protected]>

[vnet-vxlan]: Add support for scaled test

4cfa274

Signed-off-by: Lawrence Lee <[email protected]>

theasianpianist requested review from daall and prsunny August 18, 2020 02:55

theasianpianist added 4 commits August 18, 2020 12:44

[vnet_vxlan]: Move skip_cleanup fixture to conftest.py

b261115

* Allow multiple modules to access this fixture Signed-off-by: Lawrence Lee <[email protected]>

[vnet-vxlan]: Cleanup VxLAN tunnels after test

ebb0834

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Update method documentation

c9a5ca5

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Refactor global vars and helper methods

15b2f95

Signed-off-by: Lawrence Lee <[email protected]>

daall reviewed Aug 20, 2020

View reviewed changes

prsunny reviewed Aug 20, 2020

View reviewed changes

daall requested a review from a team August 20, 2020 16:34

theasianpianist added 2 commits August 20, 2020 19:00

[vnet_vxlan]: Refactor test options

3978fdc

* Move parser addoption method to test specific conftest.py * Bundle parameters for scaled vnet test into dictionary * Bundle general vnet test parameters into dictionary Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Create vnet_constants file

0e09d77

* Change all imports to match * Create vnet_config fixture for value that was previously in constants file Signed-off-by: Lawrence Lee <[email protected]>

wangxin reviewed Aug 21, 2020

View reviewed changes

tests/conftest.py Outdated Show resolved Hide resolved

tests/conftest.py Outdated Show resolved Hide resolved

tests/templates/vnet_config.j2 Show resolved Hide resolved

tests/vxlan/test_vnet_vxlan.py Show resolved Hide resolved

theasianpianist added 3 commits August 21, 2020 09:50

[vnet_vxlan]: Address review comments

9bdeaf8

* Return instead of yield in setup function * Make variable name clearer * Add pytest topology marker Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Update documentation to match style guide

0b33a46

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Move vnet templates to test specific folder

0bf7d59

Signed-off-by: Lawrence Lee <[email protected]>

theasianpianist force-pushed the vnet-vxlan-pytest branch from c6206cd to 19dc59d Compare August 21, 2020 19:18

[vnet_vxlan]: Address LGTM errors

e637822

Signed-off-by: Lawrence Lee <[email protected]>

theasianpianist force-pushed the vnet-vxlan-pytest branch from 19dc59d to e637822 Compare August 21, 2020 19:48

daall reviewed Aug 21, 2020

View reviewed changes

theasianpianist and others added 5 commits August 21, 2020 13:02

Fix typo in tests/vxlan/__init__.py

bc11655

Co-authored-by: Danny Allen <[email protected]>

Fix typo in tests/vxlan/__init__.py

3a60c2e

Co-authored-by: Danny Allen <[email protected]>

[vnet_vxlan]: Update method documentation for new CLI parameter

599125a

Signed-off-by: Lawrence Lee <[email protected]>

Merge branch 'vnet-vxlan-pytest' of https://github.com/theasianpianis…

19a70b2

…t/sonic-mgmt into vnet-vxlan-pytest

[vnet_vxlan]: Wrap calls to open using with statement

b3345f4

Signed-off-by: Lawrence Lee <[email protected]>

theasianpianist added 2 commits August 24, 2020 15:24

[vnet_vxlan]: Rewrite combine_dicts method for clarity

90a1564

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Use creds fixture to access secrets

64f3810

Signed-off-by: Lawrence Lee <[email protected]>

theasianpianist requested review from daall, prsunny and wangxin August 24, 2020 22:31

theasianpianist added 6 commits August 24, 2020 19:52

[vnet_vxlan]: Use redis-cli wrapper for database operations

fc97dc8

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Rename apply config CLI parameter for consistency

26f7709

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Refactor VNET route cleanup into separate method

b6a7127

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Remove unused variables

96f6548

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Remove unused imports/vars and fix formatting

3d5c6bf

Signed-off-by: Lawrence Lee <[email protected]>

[vnet_vxlan]: Fix --skip_apply_config behavior

780cb26

Signed-off-by: Lawrence Lee <[email protected]>

theasianpianist added the Pytest 🐍 label Aug 26, 2020

daall approved these changes Aug 27, 2020

View reviewed changes

tests/vxlan/__init__.py Outdated Show resolved Hide resolved

tests/vxlan/vnet_utils.py Outdated Show resolved Hide resolved

[vnet_vxlan]: Address review comments

63d13be

* Fix typo * Update `safe_open_template` method docstring Signed-off-by: Lawrence Lee <[email protected]>

prsunny approved these changes Aug 27, 2020

View reviewed changes

tests/vxlan/vnet_utils.py Show resolved Hide resolved

theasianpianist requested a review from volodymyrsamotiy August 28, 2020 01:32

theasianpianist changed the title ~~Vnet vxlan pytest~~ Vnet vxlan pytest conversion Aug 31, 2020

theasianpianist merged commit 578442d into sonic-net:master Aug 31, 2020

theasianpianist deleted the vnet-vxlan-pytest branch August 31, 2020 19:22

Conversation

theasianpianist commented Aug 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

daall left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

prsunny Aug 20, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

prsunny commented Aug 20, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lgtm-com bot commented Aug 21, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

theasianpianist commented Aug 24, 2020

Uh oh!

prsunny commented Aug 27, 2020

Uh oh!

daall left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

theasianpianist commented Aug 18, 2020 •

edited

Loading