Support different RC in case of pre or post sanity check failed#6860
Merged
wangxin merged 1 commit intosonic-net:masterfrom Nov 24, 2022
Merged
Support different RC in case of pre or post sanity check failed#6860wangxin merged 1 commit intosonic-net:masterfrom
wangxin merged 1 commit intosonic-net:masterfrom
Conversation
PR sonic-net#6166 introduced different pytest return code in case of sanity check failed. However, the return code is the same if pre-test or post-test sanity check failed. The run_tests.sh script skips running rest of the test scripts if sanity check failed. Actually, there is a difference between pre-test and post-test sanity check failure. The pre-test sanity check has code to do recovery. Post-test sanity check does not have recovery. If pre-test sanity check still failed after recovery, there is no point to run rest of the scripts. However, if post-test sanity check failed, we should continue to run rest of the scripts because the next script may be able to recover the testbed during pre-test sanity check. This change introduced different return code for pre-test and post-test sanity check. * pre-test only sanity check failed -> 10 * post-test only sanity check failed -> 11 * both pre and post test sanity check failed -> 12. The run_tests.sh script was updated to skip tests only when pytest return code is either 10 or 12. Signed-off-by: Xin Wang <[email protected]>
ganglyu
approved these changes
Nov 24, 2022
wangxin
added a commit
that referenced
this pull request
Nov 24, 2022
PR #6166 introduced different pytest return code in case of sanity check failed. However, the return code is the same if pre-test or post-test sanity check failed. The run_tests.sh script skips running rest of the test scripts if sanity check failed. Actually, there is a difference between pre-test and post-test sanity check failure. The pre-test sanity check has code to do recovery. Post-test sanity check does not have recovery. If pre-test sanity check still failed after recovery, there is no point to run rest of the scripts. However, if post-test sanity check failed, we should continue to run rest of the scripts because the next script may be able to recover the testbed during pre-test sanity check. This change introduced different return code for pre-test and post-test sanity check. * pre-test only sanity check failed -> 10 * post-test only sanity check failed -> 11 * both pre and post test sanity check failed -> 12. The run_tests.sh script was updated to skip tests only when pytest return code is either 10 or 12. Signed-off-by: Xin Wang <[email protected]>
wangxin
added a commit
that referenced
this pull request
Nov 24, 2022
PR #6166 introduced different pytest return code in case of sanity check failed. However, the return code is the same if pre-test or post-test sanity check failed. The run_tests.sh script skips running rest of the test scripts if sanity check failed. Actually, there is a difference between pre-test and post-test sanity check failure. The pre-test sanity check has code to do recovery. Post-test sanity check does not have recovery. If pre-test sanity check still failed after recovery, there is no point to run rest of the scripts. However, if post-test sanity check failed, we should continue to run rest of the scripts because the next script may be able to recover the testbed during pre-test sanity check. This change introduced different return code for pre-test and post-test sanity check. * pre-test only sanity check failed -> 10 * post-test only sanity check failed -> 11 * both pre and post test sanity check failed -> 12. The run_tests.sh script was updated to skip tests only when pytest return code is either 10 or 12. Signed-off-by: Xin Wang <[email protected]>
wangxin
added a commit
to wangxin/sonic-mgmt
that referenced
this pull request
Nov 29, 2022
PR sonic-net#6860 added support of return different return code in post sanity check. However, that PR introduced a typo in the run_tests.sh. This change is to fix the typo. Signed-off-by: Xin Wang <[email protected]>
6 tasks
wangxin
added a commit
that referenced
this pull request
Nov 30, 2022
PR #6860 added support of return different return code in post sanity check. However, that PR introduced a typo in the run_tests.sh. This change is to fix the typo. Signed-off-by: Xin Wang <[email protected]>
wangxin
added a commit
that referenced
this pull request
Nov 30, 2022
PR #6860 added support of return different return code in post sanity check. However, that PR introduced a typo in the run_tests.sh. This change is to fix the typo. Signed-off-by: Xin Wang <[email protected]>
wangxin
added a commit
that referenced
this pull request
Nov 30, 2022
PR #6860 added support of return different return code in post sanity check. However, that PR introduced a typo in the run_tests.sh. This change is to fix the typo. Signed-off-by: Xin Wang <[email protected]>
bingwang-ms
pushed a commit
to bingwang-ms/sonic-mgmt
that referenced
this pull request
Jul 27, 2023
…ic-mgmt into internal-202205 Fix merge conflicts. - Fix verify_no_packet_any call in fib_test (sonic-net#6461) - Fix the test case test_TSA failure when check the routes on the eos host (sonic-net#6483) - Use conditional mark to skip testcase instead of required_mocked_dualtor (sonic-net#6766) - [tagged_arp] fix issue 'fixture ports_list not found' (sonic-net#6773) - [QoS] fixes after moving to python3 (sonic-net#6786) - update parse funciton for image url (sonic-net#6848) - Fix typo in get_queue_counter (sonic-net#6852) - Revert "Fix loganalyzer.py UnicodeDecodeError (sonic-net#6524)" (sonic-net#6858) - Enhancing core_dump_and_config_check to be multi-asic aware (sonic-net#6527) - Adding support for calculating balancing in multi-lc/multi-asic case (Test_fib.py) (sonic-net#6391) - Support different RC in case of pre or post sanity check failed (sonic-net#6860) - Update getbuild.py to support pass an empty access_token - [202205] Fixing auto_techsupport (sonic-net#6882) - Merge branch 'azure-202205' into dev/yaqiangzhu/202205_manually_merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Fixes # (issue)
Type of change
Back port request
Approach
What is the motivation for this PR?
PR #6166 introduced different pytest return code in case of sanity check failed. However, the return code is the same if pre-test or post-test sanity check failed.
The run_tests.sh script skips running rest of the test scripts if sanity check failed.
Actually, there is a difference between pre-test and post-test sanity check failure. The pre-test sanity check has code to do recovery. Post-test sanity check does not have recovery. If pre-test sanity check still failed after recovery, there is no point to run rest of the scripts. However, if post-test sanity check failed, we should continue to run rest of the scripts because the next script may be able to recover the testbed during pre-test sanity check.
How did you do it?
This change introduced different return code for pre-test and post-test sanity check.
The run_tests.sh script was updated to skip tests only when pytest return code is either 10 or 12.
How did you verify/test it?
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation