Skip to content

Move skip check from test script to tests_mark_conditions.yaml for some dualtor cases#6768

Closed
ZhaohuiS wants to merge 1 commit intosonic-net:masterfrom
ZhaohuiS:guard/move_skipped_dualtor_cases
Closed

Move skip check from test script to tests_mark_conditions.yaml for some dualtor cases#6768
ZhaohuiS wants to merge 1 commit intosonic-net:masterfrom
ZhaohuiS:guard/move_skipped_dualtor_cases

Conversation

@ZhaohuiS
Copy link
Contributor

@ZhaohuiS ZhaohuiS commented Nov 8, 2022

Signed-off-by: Zhaohui Sun zhaohuisun@microsoft.com

Description of PR

Summary:
Fixes # (issue)

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 201911
  • 202012
  • 202205

Approach

What is the motivation for this PR?

Some test cases under dualtor are skipped on real dualtor testbed, but it still cost more than 10 mins to run fixtures and skip them.
Move these cases from script to tests_mark_conditions.yaml for saving time.

2022-10-27T11:03:36.5388747Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream[ipv4] SKIPPED [  8%]
2022-10-27T11:03:54.5874265Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream[ipv6] SKIPPED [ 16%]
2022-10-27T11:04:13.7323174Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream_t1_link_recovered[ipv4] SKIPPED [ 25%]
2022-10-27T11:04:32.4420177Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream_t1_link_recovered[ipv6] SKIPPED [ 33%]
2022-10-27T11:04:50.8079190Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream_bgp_recovered[ipv4] SKIPPED [ 41%]
2022-10-27T11:05:08.9424230Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream_bgp_recovered[ipv6] SKIPPED [ 50%]
2022-10-27T11:17:07.0091078Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream_loopback_route_readded[ipv4] PASSED [ 58%]
2022-10-27T11:29:54.7414890Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_downstream_loopback_route_readded[ipv6] PASSED [ 66%]
2022-10-27T11:30:21.0160615Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_remove_neighbor_downstream_standby[ipv4] SKIPPED [ 75%]
2022-10-27T11:30:36.7692504Z dualtor/test_orchagent_standby_tor_downstream.py::test_standby_tor_remove_neighbor_downstream_standby[ipv6] SKIPPED [ 83%]
2022-10-27T11:30:52.1305547Z dualtor/test_orchagent_standby_tor_downstream.py::test_downstream_standby_mux_toggle_active[ipv4] SKIPPED [ 91%]
2022-10-27T11:31:18.5181611Z dualtor/test_orchagent_standby_tor_downstream.py::test_downstream_standby_mux_toggle_active[ipv6] SKIPPED [100%]

How did you do it?

Move these skipped cases:

dualtor.test_orch_stress.test_change_mux_state
dualtor.test_orch_stress.test_flap_neighbor_entry_active
dualtor.test_orch_stress.test_flap_neighbor_entry_standby
dualtor.test_orchagent_active_tor_downstream.test_active_tor_remove_neighbor_downstream_active[ipv4]
dualtor.test_orchagent_active_tor_downstream.test_active_tor_remove_neighbor_downstream_active[ipv6]
dualtor.test_orchagent_active_tor_downstream.test_downstream_ecmp_nexthops[ipv4]
dualtor.test_orchagent_active_tor_downstream.test_downstream_ecmp_nexthops[ipv6]
dualtor.test_orchagent_mac_move.test_mac_move
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_downstream[ipv4]
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_downstream[ipv6]
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_downstream_bgp_recovered[ipv4]
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_downstream_bgp_recovered[ipv6]
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_downstream_t1_link_recovered[ipv4]
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_downstream_t1_link_recovered[ipv6]
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_remove_neighbor_downstream_standby[ipv4]
dualtor.test_orchagent_standby_tor_downstream.test_standby_tor_remove_neighbor_downstream_standby[ipv6]
dualtor.test_standby_tor_upstream_mux_toggle.test_standby_tor_upstream_mux_toggle

How did you verify/test it?

Run related dualtor cases

dualtor/test_orch_stress.py::test_flap_neighbor_entry_standby SKIPPED                                                                                                      [ 33%]
dualtor/test_orch_stress.py::test_flap_neighbor_entry_active SKIPPED                                                                                            [ 66%]
dualtor/test_orch_stress.py::test_flap_neighbor_entry_standby SKIPPED                                                                                           [100%]

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

…me dualtor cases

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
@azure-pipelines
Copy link

The pre-commit check detected issues in the files touched by this pull request.
The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame
author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...............................................................Failed
- hook id: check-yaml
- exit code: 1

while constructing a mapping
in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 823, column 3
found duplicate key "skip" with value "{}" (original value: "{}")
in "tests/common/plugins/conditional_mark/tests_mark_conditions.yaml", line 827, column 3

To suppress this check see:
http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

check for added large files..............................................Passed
check python ast.....................................(no files to check)Skipped
flake8...............................................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@ZhaohuiS
Copy link
Contributor Author

ZhaohuiS commented Nov 8, 2022

Close it since #6766 covered it.

@ZhaohuiS ZhaohuiS closed this Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant