Skip to content

Add loopback action test cases#5871

Merged
wangxin merged 1 commit intosonic-net:masterfrom
nhe-NV:add_loopback_action_tests
Feb 8, 2023
Merged

Add loopback action test cases#5871
wangxin merged 1 commit intosonic-net:masterfrom
nhe-NV:add_loopback_action_tests

Conversation

@nhe-NV
Copy link
Copy Markdown
Contributor

@nhe-NV nhe-NV commented Jun 24, 2022

Description of PR

Summary: Add test cases for loopback action
Fixes # (issue)

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 201911
  • 202012

Approach

What is the motivation for this PR?

Add new test cases for loopback action

How did you do it?

Add 3 testcases for the loopback action feature:
test_loopback_action_basic
test_loopback_action_port_flap
test_loopback_action_reload

How did you verify/test it?

Run all the new test cases, and tests pass

Any platform specific information?

No

Supported testbed topology if it's a new test case?

any

Documentation

https://github.com/sonic-net/SONiC/blob/master/doc/ip-interface/loopback-action/rif_loopback_action_testplan.md
https://github.com/sonic-net/SONiC/blob/master/doc/ip-interface/loopback-action/ip-interface-loopback-action-design.md

@nhe-NV nhe-NV requested a review from a team as a code owner June 24, 2022 14:42
@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from 0f2fc0d to 457b543 Compare June 24, 2022 14:54
@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch 2 times, most recently from 62d207c to 0cd7e83 Compare July 2, 2022 09:23
@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Jul 2, 2022

This pull request introduces 1 alert when merging 0cd7e83b19ebafea906e0219022403e58733cf5e into 23c0aee - view on LGTM.com

new alerts:

  • 1 for Unused import

@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from 0cd7e83 to cf7f747 Compare July 4, 2022 02:29
@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Jul 4, 2022

This pull request introduces 1 alert when merging cf7f747d4353f348e460436f2c8871e42f6cc2b6 into 6da07f8 - view on LGTM.com

new alerts:

  • 1 for Unused import

@liat-grozovik
Copy link
Copy Markdown
Collaborator

@nhe-NV could you please handle LGTM new alerts and build failures?

@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Jul 9, 2022

This pull request introduces 1 alert when merging b5ebcf29c911ee533218102d45b9bd0b45fb812d into ee04112 - view on LGTM.com

new alerts:

  • 1 for Unused import

@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from b5ebcf2 to 192c2e9 Compare July 14, 2022 02:11
@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Jul 14, 2022

This pull request introduces 1 alert when merging 192c2e93744e596409747f94365e89d5f31a8d0e into 7ae74e4 - view on LGTM.com

new alerts:

  • 1 for Unused import

Copy link
Copy Markdown
Contributor

@roy-sror roy-sror left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All in all looking good, Nana. Please review the minor comments. What is the test runtime?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well done, we should start using it for any new test, please inform the team

Copy link
Copy Markdown
Contributor Author

@nhe-NV nhe-NV Jul 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All in all looking good, Nana. Please review the minor comments. What is the test runtime?

It takes around 15 mins to finish all the test cases

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove_ori_dut_port_config

orig sounds more natural to me than ori

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove_ori_dut_port_config

orig sounds more natural to me than ori

Fixed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why wasn't it merged with 'setup' fixture?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it can not be merged, if merged together, then teardown for the fixture backup_and_restore_config_db_package will be executed late then the recover_config, it is not expected.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

action

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@nhe-NV
Copy link
Copy Markdown
Contributor Author

nhe-NV commented Jul 18, 2022

This pull request introduces 1 alert when merging 192c2e9 into 7ae74e4 - view on LGTM.com

new alerts:

  • 1 for Unused import

Hi, I have added "# lgtm[py/unused-import]" for the fixture which is used in the conftest.py, but is detected as "Unused import" , but it still pop up this alerts.

@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from 192c2e9 to a124695 Compare July 21, 2022 04:37
@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Jul 21, 2022

This pull request introduces 1 alert when merging a1246958b307a0237a9263f99243b4982663203c into b1866a1 - view on LGTM.com

new alerts:

  • 1 for Unused import

@liat-grozovik
Copy link
Copy Markdown
Collaborator

@roysr-nv could you please help to review following the changes requested?

@liat-grozovik
Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-mgmt

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@liat-grozovik
Copy link
Copy Markdown
Collaborator

/easycla

@liat-grozovik
Copy link
Copy Markdown
Collaborator

@roysr-nv please review

@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from a124695 to 006b8d6 Compare August 22, 2022 04:56
@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Aug 22, 2022

This pull request introduces 1 alert when merging 006b8d6339f3cadd8f1b8cfc5e96b0e7697e1497 into 1d50696 - view on LGTM.com

new alerts:

  • 1 for Unused import

roy-sror
roy-sror previously approved these changes Aug 25, 2022
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo Noe -> None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider add return None to make it more clear.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo Noe -> None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider add return None to make it more clear.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider use a define for port types.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have modules that contain CLI commands implementations that we can import from?
Same comment for the following 10 functions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find such module

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider use splitlines() instead of split("\n")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to change to show_and_parse_loopback_action

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"expect action" -> "expected action"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider change ori_ports_configuration to orig_ports_configuration

@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Aug 31, 2022

This pull request introduces 1 alert when merging e5230654406aec7d55606c8b41838491feb6267c into 976bb15 - view on LGTM.com

new alerts:

  • 1 for Unused import

@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from e523065 to a505ea3 Compare September 7, 2022 11:39
@nhe-NV
Copy link
Copy Markdown
Contributor Author

nhe-NV commented Oct 12, 2022

/azpw run

@mssonicbld
Copy link
Copy Markdown
Collaborator

/AzurePipelines run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@azure-pipelines
Copy link
Copy Markdown

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing tests/common/devices/sonic.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
autopep8.................................................................Failed
- hook id: autopep8
- files were modified by this hook
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/common/devices/sonic.py:219:121: E501 line too long (133 > 120 characters)
tests/common/devices/sonic.py:223:121: E501 line too long (132 > 120 characters)
tests/common/devices/sonic.py:247:9: E722 do not use bare 'except'
tests/common/devices/sonic.py:256:121: E501 line too long (129 > 120 characters)
tests/common/devices/sonic.py:385:54: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:385:56: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:385:72: W605 invalid escape sequence '}'
tests/common/devices/sonic.py:385:74: W605 invalid escape sequence '}'
tests/common/devices/sonic.py:390:9: E722 do not use bare 'except'
tests/common/devices/sonic.py:423:80: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:423:82: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:423:90: W605 invalid escape sequence '}'
tests/common/devices/sonic.py:423:92: W605 invalid escape sequence '}'
tests/common/devices/sonic.py:582:27: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
tests/common/devices/sonic.py:588:22: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
tests/common/devices/sonic.py:596:13: E741 ambiguous variable name 'l'
tests/common/devices/sonic.py:597:47: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:649:46: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:698:121: E501 line too long (157 > 120 characters)
tests/common/devices/sonic.py:703:121: E501 line too long (131 > 120 characters)
tests/common/devices/sonic.py:729:121: E501 line too long (145 > 120 characters)
tests/common/devices/sonic.py:843:9: E722 do not use bare 'except'
tests/common/devices/sonic.py:1028:121: E501 line too long (144 > 120 characters)
tests/common/devices/sonic.py:1040:121: E501 line too long (253 > 120 characters)
tests/common/devices/sonic.py:1063:121: E501 line too long (251 > 120 characters)
tests/common/devices/sonic.py:1079:121: E501 line too long (253 > 120 characters)
tests/common/devices/sonic.py:1096:121: E501 line too long (251 > 120 characters)
tests/common/devices/sonic.py:1131:58: F821 undefined name 'unicode'
tests/common/devices/sonic.py:1133:58: F821 undefined name 'unicode'
tests/common/devices/sonic.py:1135:58: F821 undefined name 'unicode'
tests/common/devices/sonic.py:1138:17: E741 ambiguous variable name 'l'
tests/common/devices/sonic.py:1141:69: F821 undefined name 'unicode'
tests/common/devices/sonic.py:1141:91: F821 undefined name 'unicode'
tests/common/devices/sonic.py:1150:29: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:1150:35: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:1150:39: W605 invalid escape sequence 'S'
tests/common/devices/sonic.py:1150:43: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:1150:51: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:1150:55: W605 invalid escape sequence 'S'
tests/common/devices/sonic.py:1150:59: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:1150:67: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:1150:71: W605 invalid escape sequence 'S'
tests/common/devices/sonic.py:1150:75: W605 invalid escape sequence 's'
tests/common/devices/sonic.py:1281:121: E501 line too long (121 > 120 characters)
tests/common/devices/sonic.py:1407:121: E501 line too long (135 > 120 characters)
tests/common/devices/sonic.py:1408:121: E501 line too long (135 > 120 characters)
tests/common/devices/sonic.py:1409:121: E501 line too long (135 > 120 characters)
tests/common/devices/sonic.py:1410:121: E501 line too long (135 > 120 characters)
tests/common/devices/sonic.py:1411:121: E501 line too long (135 > 120 characters)
tests/common/devices/sonic.py:1433:121: E501 line too long (972 > 120 characters)
tests/common/devices/sonic.py:1821:67: W605 invalid escape sequence '*'
tests/common/devices/sonic.py:1983:25: E127 continuation line over-indented for visual indent
tests/common/devices/sonic.py:1984:21: E124 closing bracket does not match visual indentation
tests/iface_loopback_action/conftest.py:4:121: E501 line too long (151 > 120 characters)
tests/iface_loopback_action/conftest.py:6:1: F401 'tests.common.fixtures.duthost_utils.backup_and_restore_config_db_package' imported but unused
tests/iface_loopback_action/conftest.py:183:76: F811 redefinition of unused 'backup_and_restore_config_db_package' from line 6
tests/iface_loopback_action/iface_loopback_action_helper.py:23:121: E501 line too long (123 > 120 characters)
tests/common/fixtures/duthost_utils.py:203:121: E501 line too long (138 > 120 characters)
tests/common/fixtures/duthost_utils.py:205:121: E501 line too long (138 > 120 characters)
tests/common/fixtures/duthost_utils.py:242:121: E501 line too long (181 > 120 characters)
tests/common/fixtures/duthost_utils.py:378:14: F821 undefined name 'xrange'
tests/common/fixtures/duthost_utils.py:409:22: E712 comparison to True should be 'if cond is True:' or 'if cond:'
tests/common/fixtures/duthost_utils.py:430:60: E712 comparison to True should be 'if cond is True:' or 'if cond:'

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@azure-pipelines
Copy link
Copy Markdown

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
$(results)

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Oct 14, 2022

This pull request fixes 3 alerts when merging 00ae02e369df2d64061b23084dd194808a24893c into 5bf8ec8 - view on LGTM.com

fixed alerts:

  • 3 for Except block handles 'BaseException'

@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch 2 times, most recently from ef47238 to c277fea Compare October 17, 2022 03:28
@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from c277fea to 526cb53 Compare October 28, 2022 10:15
@azure-pipelines
Copy link
Copy Markdown

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing tests/common/devices/sonic.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/common/devices/sonic.py:251:9: E722 do not use bare 'except'
tests/common/devices/sonic.py:389:54: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:389:56: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:389:72: W605 invalid escape sequence '}'
tests/common/devices/sonic.py:389:74: W605 invalid escape sequence '}'
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Copy link
Copy Markdown
Collaborator

@wangxin wangxin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking extra effort fixing code style issues in legacy code!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this still work if allure is not configured?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it works, and it is not the first time we use the allure in sonic-mgmt, such as in tests/ecmp/inner_hashing/test_inner_hashing_lag.py, we already used the allure.

@nhe-NV nhe-NV force-pushed the add_loopback_action_tests branch from 526cb53 to 090bc7a Compare November 8, 2022 01:50
@azure-pipelines
Copy link
Copy Markdown

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing tests/common/devices/sonic.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/common/devices/sonic.py:251:9: E722 do not use bare 'except'
tests/common/devices/sonic.py:389:54: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:389:56: W605 invalid escape sequence '{'
tests/common/devices/sonic.py:389:72: W605 invalid escape sequence '}'
tests/common/devices/sonic.py:389:74: W605 invalid escape sequence '}'
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@zhangyanzhao
Copy link
Copy Markdown
Contributor

waiting for build success. @wangxin @roysr-nv @liorghub can you please approve this PR if the review is ok? We need merge this PR to get the feature into 202211 release. Thanks.

roy-sror
roy-sror previously approved these changes Nov 13, 2022
@nhe-NV nhe-NV requested review from roy-sror and wangxin and removed request for wangxin December 14, 2022 03:12
Change-Id: I2f666a0ea91b3de80909362196452681e82680bc
@wangxin wangxin merged commit 77ddd3c into sonic-net:master Feb 8, 2023
ZhaohuiS added a commit that referenced this pull request Feb 28, 2023
iface_loopback_action folder was new added in #5871.
It uses package level fixture which will run the conftest before sanity check.
In ports_configuration it did some remove vlan member or portchannel member operation, which will cause the following sanity check failure obviously.

What is the motivation for this PR?
Make sanity check run before conftest of iface loopback action.

How did you do it?
Use module level fixture instead.

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
ZhaohuiS pushed a commit that referenced this pull request Mar 13, 2023
Summary: Removed git conflict records that have been merged into tests/common/devices/sonic.py(introduced by #5871
Fixes # (issue) Fix #7666
kellyyeh pushed a commit to kellyyeh/sonic-mgmt that referenced this pull request Mar 31, 2023
What is the motivation for this PR?
Add new test cases for loopback action

How did you do it?
Add 3 testcases for the loopback action feature:
test_loopback_action_basic
test_loopback_action_port_flap
test_loopback_action_reload

How did you verify/test it?
Run all the new test cases, and tests pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants