Skip to content

Add conditional step retry support#2820

Draft
jonathangiber wants to merge 2 commits intogetgauge:masterfrom
jonathangiber:feature/issue-2548-step-retry-conditions
Draft

Add conditional step retry support#2820
jonathangiber wants to merge 2 commits intogetgauge:masterfrom
jonathangiber:feature/issue-2548-step-retry-conditions

Conversation

@jonathangiber
Copy link

@jonathangiber jonathangiber commented Feb 15, 2026

Fixes #2548

Summary

  • add --max-step-retries-count to configure step-level retry attempts
  • add --retry-step-on to retry failed steps only when a regex matches error message, stacktrace, or execution messages
  • keep default behavior backward compatible (--max-step-retries-count=1, no step retries)
  • validate new flags and fail fast on invalid retry regex

Implementation

  • wire new run flags in cmd/run.go and package-level execution options in cmd/cmd.go
  • extend execution flag validation in execution/execute.go
  • add retry loop in execution/stepExecutor.go with conditional matching logic
  • add tests for retry behavior, flag conflicts, and validation

Validation

  • go test ./execution ./cmd -count=1
  • go test ./... -count=1

Notes

  • scenario-level retry (--max-retries-count) is unchanged
  • step retries stop when step passes, retry budget is exhausted, or SkipScenario is returned

- add --max-step-retries-count and --retry-step-on flags

- retry failed steps conditionally by regex over error/stack/messages

- validate new flags and add execution/cmd tests

Signed-off-by: jonathangiber <jonathangiber@gmail.com>
@github-actions
Copy link
Contributor

github-actions bot commented Feb 16, 2026

Benchmark Results

java_simple_multithreaded.csv

Commit CPU Memory Time ExitCode
d4da205 35% 66164 0:12.09 0
1b53043 33% 65936 0:12.76 0
4806ca1 35% 68112 0:11.73 0
7d7b36a 34% 68296 0:11.70 0

java_maven_parallel.csv

Commit CPU Memory Time ExitCode
d4da205 37% 162144 0:33.79 0
1b53043 36% 164988 0:35.17 0
4806ca1 33% 161880 0:37.63 0
7d7b36a 36% 161456 0:36.12 0

java_maven_multithreaded.csv

Commit CPU Memory Time ExitCode
d4da205 59% 159120 0:18.62 0
1b53043 59% 160724 0:18.07 0
4806ca1 51% 161112 0:19.41 0
7d7b36a 53% 162532 0:20.05 0

java_gradle_parallel.csv

Commit CPU Memory Time ExitCode
d4da205 125% 555352 0:41.44 0
1b53043 114% 551796 0:43.94 0
4806ca1 122% 546804 0:42.68 0
7d7b36a 121% 560676 0:40.04 0

java_maven_serial.csv

Commit CPU Memory Time ExitCode
d4da205 63% 163960 0:22.99 0
1b53043 69% 161784 0:20.27 0
4806ca1 69% 164512 0:20.22 0
7d7b36a 66% 161936 0:21.82 0

java_gradle_multithreaded.csv

Commit CPU Memory Time ExitCode
d4da205 201% 555408 0:24.89 0
1b53043 199% 558756 0:22.68 0
4806ca1 200% 536500 0:25.63 0
7d7b36a 207% 551276 0:24.61 0

java_gradle_serial.csv

Commit CPU Memory Time ExitCode
d4da205 198% 562236 0:27.18 0
1b53043 198% 558020 0:26.77 0
4806ca1 194% 551996 0:27.21 0
7d7b36a 196% 557248 0:27.17 0

java_simple_serial.csv

Commit CPU Memory Time ExitCode
d4da205 43% 68148 0:17.14 0
1b53043 49% 68088 0:13.81 0
4806ca1 42% 66320 0:12.22 0
7d7b36a 39% 68108 0:13.88 0

java_simple_parallel.csv

Commit CPU Memory Time ExitCode
d4da205 24% 68000 0:23.57 0
1b53043 23% 66252 0:24.28 0
4806ca1 23% 68096 0:24.86 0
7d7b36a 23% 68172 0:23.72 0

Notes

  • The results above are generated by running against seed projects in https://github.com/getgauge/gauge-benchmark
  • These results are not persisted, but on merging to master the benchmark will be rerun.
  • These benchmark are run in Github Actions' agents, which are virtualized. Results are not to be taken as actual benchmarks.Rather, these are indicative numbers and make sense for comparison.

See Workflow log for more details.

@jonathangiber
Copy link
Author

@chadlwilson any change to go over this one?

@chadlwilson
Copy link
Contributor

@chadlwilson any change to go over this one?

It's not an area I am familiar with, nor something I have a personal view on whether is a good idea in the first place.

Also not sure if a philosophy was agreed on how far the tool should go to handle retries/flaky tests, since I am really just a "minor maintenance guy" and wasn't involved with the fundamental design of Gauge.

Perhaps @sriv has a view, but he's probably rather busy.

@sriv
Copy link
Member

sriv commented Mar 3, 2026

I am personally not a fan of retries. They often hide underlying issues, and a green run could mask issues.

But this has been something many folks have asked for, and we ended up building it. This additional sophistication of conditional retry makes me believe even more that this is not a good idea. But gauge has become less opinionated over time. Don't want to debate the utility of this feature here.

That said, in this PR I am not a fan of regex matching of error patterns that would trigger a decision. It can cause false positives. If we were to do this, i would consider something more deterministic like an error code or strongly typed exception classes etc.

@chadlwilson chadlwilson marked this pull request as draft March 13, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Step Retry Mechanism with Conditional Logic

3 participants