Fix conflicts in ansible tmp dir for parallel run#2356
Merged
wangxin merged 2 commits intosonic-net:masterfrom Oct 21, 2020
wangxin:parallel-tmp-dir
Merged
Fix conflicts in ansible tmp dir for parallel run#2356wangxin merged 2 commits intosonic-net:masterfrom wangxin:parallel-tmp-dir
wangxin merged 2 commits intosonic-net:masterfrom
wangxin:parallel-tmp-dir
Conversation
Parallel execution by multiprocessing is used in bgp/conftest.py for configuring multiple VM nodes. However, all the subprocesses are using same tmp dir for `ansiballz` related operation. There could be conflicts. This change added a decorator for resetting ansible tmp folder for each subprocess. Then each subprocess has its own unique tmp folder. Signed-off-by: Xin Wang <xiwang5@microsoft.com>
lolyu
reviewed
Oct 16, 2020
|
This pull request introduces 1 alert when merging 282b43c into bdfd25b - view on LGTM.com new alerts:
|
yxieca
reviewed
Oct 16, 2020
tests/common/helpers/parallel.py
Outdated
| # Reset the ansible default local tmp directory for the current subprocess | ||
| # Otherwise, multiple processes could share a same ansible default tmp directory and there could be conflicts | ||
| from ansible import constants | ||
| import os, shutil, tempfile |
Collaborator
There was a problem hiding this comment.
maybe move imports to the top of the file with other imports? os was imported before and LGTM is complaining about that.
Collaborator
Author
There was a problem hiding this comment.
Moved import os, shutil, tempfile to top of file. Since importing ansible is just for updating temp folder for sub-processes, this importing is still kept here.
lguohan
approved these changes
Oct 21, 2020
kazinator-arista
pushed a commit
to kazinator-arista/sonic-mgmt
that referenced
this pull request
Mar 4, 2026
…x-kernel] advance submodule head (sonic-net#12025) linkmgrd: * ab5b2c1 2022-09-02 | Fix mux config (sonic-net#128) (HEAD -> 202205, github/202205) [Longxiang Lyu] utilities: * 7de9305 2022-09-07 | [generate dump]Added error message when saisdkdump fails (sonic-net#2356) (HEAD -> 202205, github/202205) [Sudharsan Dhamal Gopalarathnam] * c5b0a6d 2022-09-07 | [counterpoll]Fixing counterpoll show for tunnel and acl stats (sonic-net#2355) [Sudharsan Dhamal Gopalarathnam] * 1452b44 2022-09-05 | [GCU] Fix missing backend in dry run (sonic-net#2347) [jingwenxie] * bc7b845 2022-09-04 | Add Password Hardening CLI support (sonic-net#2338) [davidpil2002] * 55e8948 2022-09-06 | [fast-reboot]Avoid stopping masked services during fast-reboot (sonic-net#2335) [Sudharsan Dhamal Gopalarathnam] * f7d69d4 2022-08-30 | Replace cmp in acl_loader with operator.eq (sonic-net#2328) [Zhaohui Sun] * 4054ebb 2022-09-05 | Add verification for override (sonic-net#2305) [jingwenxie] * 729d811 2022-05-30 | Fix sonic-installer and 'show version' command crash when database docker not running issue. (sonic-net#2183) [Hua Liu] platform-daemons: * 36ba7c0 2022-09-07 | [ycable] cleanup logic for creating grpc future ready (sonic-net#289) (HEAD -> 202205) [vdahiya12] * 2a9db73 2022-09-01 | [ycabled] fix insert events from xcvrd;cleanup some mux toggle logic (sonic-net#287) [vdahiya12] platform-common: * d7c990d 2022-09-03 | [CMIS] 'get_transceiver_info' should return 'None' when CMIS cable EEPROM is not ready (sonic-net#305) (HEAD -> 202205) [Kebo Liu] linux-kernel: * 25ea052 2022-08-31 | [patch]: Add accpt_untracked_na kernel param (sonic-net#292) (HEAD -> 202205) [Lawrence Lee] Signed-off-by: Ying Xie <ying.xie@microsoft.com> Signed-off-by: Ying Xie <ying.xie@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Fixes # (issue)
Script
test_bgp_gr_helper.pyoccasionally failed with:This is because of conflicts in parallel run for configuring multiple VMs in
bgp/conftest.py. This PR is to fix the conflict issue.Type of change
Approach
What is the motivation for this PR?
Parallel execution by multiprocessing is used in bgp/conftest.py for
configuring multiple VM nodes. However, all the subprocesses are
using same tmp dir for
ansiballzrelated operation. There could beconflicts.
How did you do it?
This change added a decorator for resetting ansible tmp folder for each subprocess. Then each subprocess has its own unique tmp folder.
How did you verify/test it?
Used temp debug code to verify that each subprocess is using unique temp dir.
Verified that temp directories are cleared after test.
Test run
bgp/test_bgp_gr_helper.pymultiple times.Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation