Skip to content

tool to bisect culprit commits that cause test failure or regression. #8449

@panyx0718

Description

@panyx0718

It's common to write commits that are found to be bad long after the merge:

  1. Flaky test due to random inputs, multi-threading or distribution.
  2. Changes that cause performance or convergence regression.
  3. etc.

We need a tool to automatically bisect to the culprit commit.
The tool should:

  1. binary search over the commits
  2. for each selected commits, evaluate if it's good or bad.
  3. decide the first commit causing the problem.

Notes:
It's common to merge a PR that contains several commits. Only the merge commit is tested to be good by the CI before merge. Hence, we only bisect the commits that merged to the mainline (e.g. develop) branch.

Advanced Feature:
The CI should continuously evaluate recent commits and record the history. This allows developers to quickly find out when does the problem start.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions