Skip to content

Configure VMs in parallel for test_bgp_gr_helper#2280

Merged
wangxin merged 3 commits intosonic-net:masterfrom
wangxin:bgp_gr_parallel
Oct 10, 2020
Merged

Configure VMs in parallel for test_bgp_gr_helper#2280
wangxin merged 3 commits intosonic-net:masterfrom
wangxin:bgp_gr_parallel

Conversation

@wangxin
Copy link
Copy Markdown
Collaborator

@wangxin wangxin commented Sep 28, 2020

Description of PR

Summary:
Fixes # (issue)
Add a helper function to run a function in parallel for multiple nodes. Update the test_bgp_gr_helper test to use this new function to configure VM nodes in parallel.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Approach

What is the motivation for this PR?

The test_bgp_gr_helper.py script needs to configure each of the VM nodes one by one.
For topologies like t1 and t1-lag, it takes a long time to do that. This change

How did you do it?

  • Added a helper function to run the configuration tasks in parallel using the python
    multiprocessing library.
  • Updated the setup code for test_bgp_gr_helper to configure the VMs in parallel.

Without this change, the test_bgp_gr_helper.py script needs over 900 seconds to run
on t1-lag topology. With this change, testing time is decreased to around 200 seconds

How did you verify/test it?

Test run test_bgp_gr_helper using topologies t0 and t1-lag.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

@wangxin
Configure VMs in parallel for test_bgp_gr_helper
48870bf 
The test_bgp_gr_helper.py script needs to configure each of the VM nodes one by one.
For topologies like t1 and t1-lag, it takes a long time to do that. This change
* Added a helper function to run the configuration tasks in parallel using the python
  multiprocessing library.
* Updated the setup code for test_bgp_gr_helper to configure the VMs in parallel.

Without this change, the test_bgp_gr_helper.py script needs over 900 seconds to run
on t1-lag topology. With this change, testing time is decreased to around 200 seconds

Signed-off-by: Xin Wang <[email protected]>
@wangxin wangxin requested a review from a team September 28, 2020 06:57
@lgtm-com
Copy link
Copy Markdown

lgtm-com bot commented Sep 28, 2020

This pull request introduces 1 alert when merging 48870bf into d190bb2 - view on LGTM.com

new alerts:

  • 1 for Except block handles 'BaseException'

@wangxin
Copy link
Copy Markdown
Collaborator Author

wangxin commented Sep 30, 2020

retest this please

lguohan
lguohan reviewed Oct 2, 2020
View reviewed changes
tests/bgp/conftest.py Outdated
failed_results[node_name] = failed_node_results
if failed_results:
logger.error('failed_results => {}'.format(json.dumps(failed_results, indent=2)))
pt_assert(False, 'Some processes for configuring nbr hosts returned failed results')
Copy link
Copy Markdown
Contributor

@lguohan lguohan Oct 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line 64-71 is most duplicated in two places, wonder if you can provide helper functions to ease the result check.

Copy link
Copy Markdown
Collaborator Author

@wangxin wangxin Oct 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, updated.

@wangxin wangxin merged commit 457a640 into sonic-net:master Oct 10, 2020
@wangxin wangxin deleted the bgp_gr_parallel branch November 30, 2020 09:56
kazinator-arista pushed a commit to kazinator-arista/sonic-mgmt that referenced this pull request Mar 4, 2026
@yxieca
[202205][linkmgrd][utilities][swss][swss-common][sairedis][platform-d…
e8fe21b 
…aemon] advance submodule head (sonic-net#11652)

linkmgrd:
* 17240f1 2022-08-02 | [active-active] Update unhealthy label definition (sonic-net#102) (HEAD -> 202205, github/202205) [Jing Zhang]
* 99a7b4b 2022-07-28 | Update the definition of healthy label (sonic-net#99) [Jing Zhang]
* 551144b 2022-08-08 | [Active-Standby]Remove unnecessary `handleMuxWaitTimeout` logs (sonic-net#100) [Jing Zhang]

utilities:
* 8cbbe4f 2022-08-08 | [crm] add checking for CRM interval range (sonic-net#2293) (HEAD -> 202205, github/202205) [Andriy Yurkiv]
* 7dac7a7 2022-08-03 | Fix version in db_migrator  for `PORT_QOS_MAP|global` (#2289) [bingwang-ms]
* e14f0c9 2022-08-03 | Add override testcase to verify removal (#2288) [jingwenxie]
* 61ea21d 2022-07-29 | [watermarkstat] Fix CLI script for unconfigured PG counters (sonic-net#2239) [Nazarii Hnydyn]
* e206e2d 2022-05-19 | [portconfig] Allow to configure interface mtu for physical ports only [Sudharsan Dhamal Gopalarathnam]
* 9751479 2022-08-01 | Transfer organization from Azure to sonic-net (sonic-net#2285) [Liu Shilong]
* 218b175 2022-07-29 | [202205][counters] Keep counters cache in a single directory (sonic-net#2280) [Stepan Blyshchak]

swss:
* d772a70 2022-08-09 | [muxorch] return true if the nbr IP is in the skip_neighbors list (sonic-net#2407) (HEAD -> 202205) [Nikola Dancejic]
* a3ac275 2022-08-08 | [EVPN]Modified tunnel creation logic when creating tunnel in VRF-VNI map creation flow (sonic-net#2404) (HEAD -> 202205) [Sudharsan Dhamal Gopalarathnam]
* 67c1376 2022-08-09 | [QoS] Fix issue: the WRED profile can not be set if current min > new max or current max < new min (sonic-net#2379) [Stephen Sun]
* 0295a35 2022-07-21 | [Buffer Orch] Support removing buffer port profile list (sonic-net#2371) [Stephen Sun]
* 450e9ec 2022-07-07 | Fix: missing sonic-db-cli in docker-sonic-vs image (sonic-net#2357) [Qi Luo]
* c1c1147 2022-06-02 | Purge package sonic-db-cli which depends on libswsscommon (sonic-net#2308) [Qi Luo]

swss-common:
* 749cd6f 2022-07-21 | Fix PING database command not backward compatible with python version issue. (sonic-net#650) (HEAD -> 202205) [Hua Liu]
* 4fa7484 2022-06-27 | Fix: missing sonic-db-cli in docker-sonic-vs image (sonic-net#640) [Qi Luo]
* 4d0dc7a 2022-06-23 | [sonic-cli] Fix sonic-db-cli output format not backward compatible with python version issue. (sonic-net#631) [Hua Liu]
* c5573fe 2022-06-03 | Purge package sonic-db-cli which depends on libswsscommon (sonic-net#628) [Qi Luo]
* 0ade483 2022-08-02 | Add docker-mux related table names  (sonic-net#627) [Jing Zhang]
* 675d793 2022-05-26 | Re-write sonic-db-cli with c++ for sonic startup performance issue (sonic-net#607) [Hua Liu]
* 1155773 2022-08-01 | Transfer organization from Azure to sonic-net (sonic-net#661) (github/202205) [Liu Shilong]

sairedis:
* c62a453 2022-06-03 | Purge package sonic-db-cli which depends on libswsscommon (sonic-net#1057) (HEAD -> 202205) [Qi Luo]
* 8be9292 2022-07-20 | [syncd] Remove error message when tryTranslateVidToRid fails (sonic-net#1048) [Kamil Cudnik]
* eb552ee 2022-08-01 | Transfer organization from Azure to sonic-net (sonic-net#1099) (github/202205) [Liu Shilong]

platform-daemon:
* d150229 2022-08-11 | Correct the peer forwarding state table (sonic-net#281) (HEAD -> 202205) [Longxiang Lyu]
* 9507e6c 2022-08-10 | [ycabled] remove some spurious logs (sonic-net#282) (HEAD -> 202205) [vdahiya12]

Signed-off-by: Ying Xie <[email protected]>
kazinator-arista pushed a commit to kazinator-arista/sonic-mgmt that referenced this pull request Mar 4, 2026
@yxieca
[202205][utilities][swss][sairedis] advance submodule head (sonic-net…
bf890a4 
…#12974)

utilities:
* 4b51e41 2022-12-06 | [config] Add check in config interface ip command to block if the interface is portchannel member (sonic-net#2539) (HEAD -> 202205) [Sudharsan Dhamal Gopalarathnam]
* e53b32e 2022-12-06 | [generate_dump] [Mellanox] Fix the duplicate dfw dump collection problem by adding symlinks (sonic-net#2536) [Vivek]
* 0391221 2022-12-02 | [GCU] Add RemoveCreateOnlyDependency Validator/Generator (sonic-net#2500) [jingwenxie]
* e3658e9 2022-04-13 | [scripts/fast-reboot] Shutdown remaining containers through systemd (sonic-net#2133) [Stepan Blyshchak]

swss:
* 1a4a5d9 2022-12-02 | [ACL] Support ACTION_COUNTER action in custom ACL table type (sonic-net#2550) [bingwang-ms]
* 33b0a9e 2022-12-05 | [muxorch] Adding case for maintaining current state (sonic-net#2280) [Nikola Dancejic]

sairedis:
* b29bb45 2022-12-02 | enable cisco8000 SAI bulk API feature (sonic-net#1153) (HEAD -> 202205) [Keith Lu]

Signed-off-by: Ying Xie <[email protected]>

Signed-off-by: Ying Xie <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lguohan lguohan lguohan approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@wangxin @lguohan