Add conditional step retry support#2820
Add conditional step retry support#2820jonathangiber wants to merge 2 commits intogetgauge:masterfrom
Conversation
- add --max-step-retries-count and --retry-step-on flags - retry failed steps conditionally by regex over error/stack/messages - validate new flags and add execution/cmd tests Signed-off-by: jonathangiber <jonathangiber@gmail.com>
Benchmark Results
Notes
See Workflow log for more details. |
|
@chadlwilson any change to go over this one? |
It's not an area I am familiar with, nor something I have a personal view on whether is a good idea in the first place. Also not sure if a philosophy was agreed on how far the tool should go to handle retries/flaky tests, since I am really just a "minor maintenance guy" and wasn't involved with the fundamental design of Gauge. Perhaps @sriv has a view, but he's probably rather busy. |
|
I am personally not a fan of retries. They often hide underlying issues, and a green run could mask issues. But this has been something many folks have asked for, and we ended up building it. This additional sophistication of conditional retry makes me believe even more that this is not a good idea. But gauge has become less opinionated over time. Don't want to debate the utility of this feature here. That said, in this PR I am not a fan of regex matching of error patterns that would trigger a decision. It can cause false positives. If we were to do this, i would consider something more deterministic like an error code or strongly typed exception classes etc. |
Fixes #2548
Summary
--max-step-retries-countto configure step-level retry attempts--retry-step-onto retry failed steps only when a regex matches error message, stacktrace, or execution messages--max-step-retries-count=1, no step retries)Implementation
cmd/run.goand package-level execution options incmd/cmd.goexecution/execute.goexecution/stepExecutor.gowith conditional matching logicValidation
go test ./execution ./cmd -count=1go test ./... -count=1Notes
--max-retries-count) is unchangedSkipScenariois returned