Use conditional mark to skip testcase instead of required_mocked_dualtor by xwjiang-ms · Pull Request #6766 · sonic-net/sonic-mgmt

xwjiang-ms · 2022-11-08T03:49:25Z

Description of PR

Summary:
Fixes # (issue)

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

201911
202012
202205

Approach

What is the motivation for this PR?

Currently, we use required_mocked_dualtor to skip testcases in real dualtor under tests/dualtor/, but it's not efficient enough, and will cause show mux status error after toggle.

How did you do it?

Use conditional mark to skip testcase instead of required mocked dualtor

How did you verify/test it?

Run tests

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

What is the motivation for this PR? Currently, we use required_mocked_dualtor to skip testcases in real dualtor under `tests/dualtor/`, but it's not efficient enough, and will cause show mux status error after toggle. How did you do it? Use conditional mark to skip testcase instead of required mocked dualtor How did you verify/test it? Run tests Any platform specific information? Supported testbed topology if it's a new test case?

azure-pipelines · 2022-11-08T03:50:13Z

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Failed - hook id: check-yaml - exit code: 1 while constructing a mapping in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 827, column 3 found duplicate key "skip" with value "{}" (original value: "{}") in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 831, column 3 To suppress this check see: http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

lgtm-com · 2022-11-08T04:11:53Z

This pull request introduces 1 alert when merging 48c18cb into a4feb41 - view on LGTM.com

new alerts:

1 for Unused import

azure-pipelines · 2022-11-08T05:58:39Z

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Failed - hook id: check-yaml - exit code: 1 while constructing a mapping in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 827, column 3 found duplicate key "skip" with value "{}" (original value: "{}") in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 831, column 3 To suppress this check see: http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

azure-pipelines · 2022-11-08T06:55:33Z

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Failed - hook id: check-yaml - exit code: 1 while constructing a mapping in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 827, column 3 found duplicate key "skip" with value "{}" (original value: "{}") in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 831, column 3 To suppress this check see: http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

ZhaohuiS · 2022-11-08T07:17:05Z

@xwjiang-ms Since your PR covered the same issue in my PR #6768 , I will close mine.

ZhaohuiS

LGTM

tests/dualtor/test_orchagent_standby_tor_downstream.py

azure-pipelines · 2022-11-08T09:24:44Z

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Failed - hook id: check-yaml - exit code: 1 while constructing a mapping in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 833, column 3 found duplicate key "skip" with value "{}" (original value: "{}") in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 837, column 3 To suppress this check see: http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

tests/common/plugins/conditional_mark/tests_mark_conditions.yaml

azure-pipelines · 2022-11-09T14:28:38Z

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Failed - hook id: check-yaml - exit code: 1 while constructing a mapping in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 833, column 3 found duplicate key "skip" with value "{}" (original value: "{}") in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 837, column 3 To suppress this check see: http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

ZhaohuiS

LGTM

…tor (#6766) * Use conditional mark to skip testcase instead of required mocked dualtor What is the motivation for this PR? Currently, we use required_mocked_dualtor to skip testcases in real dualtor under `tests/dualtor/`, but it's not efficient enough, and will cause show mux status error after toggle. How did you do it? Use conditional mark to skip testcase instead of required mocked dualtor How did you verify/test it? Run tests Any platform specific information? Supported testbed topology if it's a new test case?

…ic-mgmt into internal-202205 Fix merge conflicts. - Fix verify_no_packet_any call in fib_test (sonic-net#6461) - Fix the test case test_TSA failure when check the routes on the eos host (sonic-net#6483) - Use conditional mark to skip testcase instead of required_mocked_dualtor (sonic-net#6766) - [tagged_arp] fix issue 'fixture ports_list not found' (sonic-net#6773) - [QoS] fixes after moving to python3 (sonic-net#6786) - update parse funciton for image url (sonic-net#6848) - Fix typo in get_queue_counter (sonic-net#6852) - Revert "Fix loganalyzer.py UnicodeDecodeError (sonic-net#6524)" (sonic-net#6858) - Enhancing core_dump_and_config_check to be multi-asic aware (sonic-net#6527) - Adding support for calculating balancing in multi-lc/multi-asic case (Test_fib.py) (sonic-net#6391) - Support different RC in case of pre or post sanity check failed (sonic-net#6860) - Update getbuild.py to support pass an empty access_token - [202205] Fixing auto_techsupport (sonic-net#6882) - Merge branch 'azure-202205' into dev/yaqiangzhu/202205_manually_merge

Fix LGTM alert

43d882e

Remove pt_require to let cases run on t0

db95ec9

ZhaohuiS previously approved these changes Nov 8, 2022

View reviewed changes

ZhaohuiS mentioned this pull request Nov 8, 2022

Move skip check from test script to tests_mark_conditions.yaml for some dualtor cases #6768

Closed

6 tasks

ZhaohuiS reviewed Nov 8, 2022

View reviewed changes

tests/dualtor/test_orchagent_standby_tor_downstream.py Show resolved Hide resolved

test_standby_tor_downstream_loopback_route_readded should only run on t0

cb5ecee

xwjiang-ms dismissed ZhaohuiS’s stale review via cb5ecee November 8, 2022 09:23

xwjiang-ms requested a review from ZhaohuiS November 9, 2022 05:44

ZhaohuiS reviewed Nov 9, 2022

View reviewed changes

tests/common/plugins/conditional_mark/tests_mark_conditions.yaml Outdated Show resolved Hide resolved

Improve skip reason

a039248

ZhaohuiS approved these changes Nov 10, 2022

View reviewed changes

xwjiang-ms merged commit 9eb6e06 into sonic-net:master Nov 10, 2022

xwjiang-ms deleted the change_mock_skip_to_conditional_mark branch November 10, 2022 01:01

xwjiang-ms added Request for 202205 branch Request for 202012 branch labels Nov 22, 2022

wangxin added the Included in 202205 branch label Nov 23, 2022

wangxin added the Included in 202012 branch label Nov 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use conditional mark to skip testcase instead of required_mocked_dualtor#6766

Use conditional mark to skip testcase instead of required_mocked_dualtor#6766
xwjiang-ms merged 5 commits intosonic-net:masterfrom
xwjiang-ms:change_mock_skip_to_conditional_mark

xwjiang-ms commented Nov 8, 2022

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

lgtm-com bot commented Nov 8, 2022

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

ZhaohuiS commented Nov 8, 2022

Uh oh!

ZhaohuiS left a comment

Uh oh!

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

Uh oh!

azure-pipelines bot commented Nov 9, 2022

Uh oh!

ZhaohuiS left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

xwjiang-ms commented Nov 8, 2022

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

lgtm-com bot commented Nov 8, 2022

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

ZhaohuiS commented Nov 8, 2022

Uh oh!

ZhaohuiS left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

azure-pipelines bot commented Nov 8, 2022

Uh oh!

Uh oh!

azure-pipelines bot commented Nov 9, 2022

Uh oh!

ZhaohuiS left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants