Encountered a case where the tests pass and the check is published, but the PR doesn't receive the update for the check. Therefore, the auto-merge doesn't work.

[tanin47]

The run seems to publish the correct check:

But it seems the PR didn't receive the new status of the check.

Here's our config:

Our PR statuses look like this:

Since the PR still waits for the status of Test results, it doesn't trigger auto-merge.

I have a few questions:

• Is there a way to debug more? I don't have further idea how to further debug this.
• What are some possible root cause for this?

Thank you.

[mikepenz]

Good day.

Is this a consistent behavior or just a random occurrence? (There was a github actions outage yesterday, so I'd like to rule that out)

[mccarron]

So we just hit the same thing just now, happening for every run for a specific PR at this time. So trying to test another with no changes to see what happens.

Also so far nothing on the Github Status page. So if there another outage, it's not wide enough to be noticed.

[mccarron]

@tanin47 What runner is this action running on? For us it's a self hosted macOS runner.

The machine where this was happening every time was updated to the latest Command Line Tools for Xcode 16.4 and the other CI machine we have that works fine does not have the updated tools installed. Not sure why it would cause this, but that's our current lead.

[mikepenz]

Just to confirm once more. As I can't make out from the current answers (frankly I'm in the phone right now, so I might missed it)

That behavior is something that started happening recently / yesterday? Or around the time of the last release? And if so, going back to the prior release fixes it? Or it's always been that way?

[tanin47]

It seems to start since about a week ago. It could be something on our side because the Github Actions yaml is fairly complex.

Currently, action-junit-report reports the check successfully as we see in the screenshot. But for some reason the PR doesn't receive the update. I don't have any idea how to debug what happens in between.

What kind of steps would you take to debug this? It might be better for me to take those steps to debug, so I can understand whether it's our setup or it's a library bug.

Thank you!

---

To answer the specific questions:

What runner is this action running on?

It's hosted on Github Actions (not self-hosted).

Is this a consistent behavior or just a random occurrence?

It happens maybe half of the times, which is perplexing. Our team cannot see any underlying pattern.

[mikepenz]

If it's a week ago. There's a chance that it's the latest release. However it happening 50% of times somehow makes it unlikely and seems kinda more likely related to maybe #40 ? 🤔

[mccarron]

@tanin47:

It's hosted on Github Actions (not self-hosted).

What type of machine / OS? I wager you are not using macOS on it?

@mikepenz:

Just to confirm once more. As I can't make out from the current answers (frankly I'm in the phone right now, so I might missed it)

On this setup it appeared to be every time, we reverted that machine and it went away.

seems kinda more likely related to maybe #40 ? 🤔

Wow yeah that's a bad one, and feels REALLY similar. Could be a streak of bad luck made it feel like every time.

I'll keep an eye out for this happening again, but I really don't think it has to do with Xcode as I wager @tanin47 was not using macOS.

[tanin47]

Oops sorry about that. I didn't answer the question. It's not Mac. It's Ubuntu.

[tanin47]

One question: How do we use v5.5.1 directly? I can try using that for a few weeks to see if the problem will still occur.

[mikepenz]

You can use any tag or commit specifically

uses: mikepenz/action-junit-report@v5.5.1

Or instead of v5.5.1 you can also specific specific commit hashes

[tanin47]

I'd like to report back. Rolling back to v5.5.1 doesn't work, but I did find out more.

1. We have the latest run #9302 and the job process_test_reports that uses your plugin to publish the check Test result.

2. I've noticed that #9302 doesn't have the Test result check in the left panel, which is odd. Because a published check should have existed. The log obviously said it was published.

3. I've noticed that we have a canceled #9301 (which has the same sha as #9302). And it has 7 published Test result check. It turns out that publishing a check went to the previous canceled run, and this is why the PR which corresponds to the latest run doesn't receive the Test result check up.

4. I deleted the #9301 workflow run from the UI. Then, I go to #9302 and trigger the re-run of process_test_reports.

The PR got the Test result check, and the auto-merge kicked in.

I still don't know why the published check went to the canceled run #9301. But now we know why it doesn't work sometimes. The check is published to a previous run that has the same sha.

I've looked back in the history. For some reason, we have spawn the first action run... then 30s we would spawn another run, which would cause the first run to be canceled. It happened quite a lot but not every time.

The mystery is resolved.

We'll find out why it spawns double-run and how we can report a check to the latest run instead of the previous run with the same sha.

---

Update: We started doing a lot of the double runs because we recently triggered the Actions run based on the pull_request.edited event, which would be triggered by editing a PR's description. This causes us to have 2 Action runs with the same sha.

It seems the real root cause is that there is no way to publish a check to the newest run if a previous run with the same sha exists.

[mikepenz]

Thank you @tanin47 for the in-depth analysis, and diving more into the problem.

So it appears that the problem is actually caused by: #40

Unfortunately, this also means that , unless there are new APIs provided to solve that, there's no way to fix it in the action at this time :/

Also I think we can close this report as a duplicate?

[tanin47]

yup thank you.

[mikepenz]

The only thing that you could maybe test is this one post here: https://github.com/orgs/community/discussions/24616#discussioncomment-5607870

but it still feels wrong in my opinion, and I'd love to have a proper api to fix this :D