[3.4] fix: linux-amd64-e2e job timeout issue #17304
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Author
|
ping @ahrtr @serathius @jmhbnz |
Member
|
Thanks for the analysis. I suggest one of the following solutions,
|
Assume etcd-io#16916 as baseline. The E2E takes `1395.082s`. * etcd-io#16988 It introduced `TestAuthority` which takes `18.39s`. And after etcd-io#16997, it takes `50.05s`. * etcd-io#16995 It introduced `TestInPlaceRecovery` which takes `17.37s`. * etcd-io#17144 - New `TestHTTPHealthHandler` takes `29.9s` - New `TestHTTPLivezReadyzHandler` takes `35.20s` * etcd-io#17173 - New `TestMemberReplace` takes `7.55s`. Ideally, it should increase `140.07s`. It's not larger than `1800s` timeout value. However, we run E2E cases 3 times. By default, we run E2E cases with `-cpu 1,2,4`. That means that we run 3 times. `1395.082s` + `140.07s * 3` = `1815.292s` > `1800s` ```bash $ go help testflag -count n Run each test, benchmark, and fuzz seed n times (default 1). If -cpu is set, run n times for each GOMAXPROCS value. Examples are always run once. -count does not apply to fuzz tests matched by -fuzz. ``` I don't think we should run E2E with different GOMAXPROCS value. All the `TestXYZ` are used to control etcd process and we don't set GOMAXPROCS env to etcd process. Set `CPU=4` to align with main and release/3.5. Closes: etcd-io#17241 Signed-off-by: Wei Fu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Assume #16916 as baseline. The E2E takes
1395.082s.It introduced
TestAuthoritywhich takes18.39s. And after #16997, it takes50.05s.It introduced
TestInPlaceRecoverywhich takes17.37s.[3.4] Backport e2e tests for livez/readyz. #17144
TestHTTPHealthHandlertakes29.9sTestHTTPLivezReadyzHandlertakes35.20s[3.4] member replace e2e test #17173
TestMemberReplacetakes7.55s.Ideally, it should increase
140.07s. It's not larger than1800stimeout value.However, we run E2E cases 3 times. By default, we run E2E cases with
-cpu 1,2,4. That means that we run 3 times.1395.082s+140.07s * 3=1815.292s>1800sI don't think we should run E2E with different GOMAXPROCS value. All the
TestXYZare used to control etcd process and we don't set GOMAXPROCS env to etcd process. So, we don't need-cpusetting for E2E.Closes: #17241
Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.