Skip test scripts in PR testing using pytest markers. #15872
Skip test scripts in PR testing using pytest markers. #15872yutongzhang-microsoft wants to merge 1 commit intosonic-net:masterfrom
Conversation
|
With this new mark, it would be easily for people to consider these scripts are physical testbed only. Considering that, hard code a mark to indicate that these scripts are physical testbed only does not really make sense to me. |
Based on prior analysis, these scripts were confirmed to run exclusively on physical testbeds and were therefore added to the skip list, excluding them from PR testing. For scripts partially compatible with PR testing, they are not in the skip list. So the scripts in skip list can be considered as 'physical testbed only'. |
Description of PR
Previously, we used the
.azure-pipelines/pr_test_skip_scripts.yamlfile to manually specify test scripts to skip during PR testing. However, in our new PR testing model proposed in #15666, test scripts are collected automatically, eliminating the need for this hardcoded file.To handle skipped scripts in the new model, we now identify them directly within the test scripts. Specifically, we use the
pytest.mark.device_type('physical')marker to indicate scripts that can only run on physical testbeds and should be skipped in PR testing. This PR adds the necessary markers to the relevant scripts to align with the new approach.Summary:
Fixes # (issue)
Type of change
Back port request
Approach
What is the motivation for this PR?
Previously, we used the
.azure-pipelines/pr_test_skip_scripts.yamlfile to manually specify test scripts to skip during PR testing. However, in our new PR testing model proposed in #15666, test scripts are collected automatically, eliminating the need for this hardcoded file.To handle skipped scripts in the new model, we now identify them directly within the test scripts. Specifically, we use the
pytest.mark.device_type('physical')marker to indicate scripts that can only run on physical testbeds and should be skipped in PR testing. This PR adds the necessary markers to the relevant scripts to align with the new approach.How did you do it?
This PR adds the necessary markers to the relevant scripts to align with the new approach.
How did you verify/test it?
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation