Skip to content

Let a shown trace layer settle before its test destroys it - #446

Open
zuurbr wants to merge 1 commit into
mainfrom
trace-layer-teardown-flake
Open

zuurbr wants to merge 1 commit into
mainfrom
trace-layer-teardown-flake

Conversation

@zuurbr

@zuurbr zuurbr commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

TraceLayerTest.FollowsTheBackingScaleWithNoSizeChange crashed twice on the macOS 15 runner, once in the merge queue. The crash is inside QuartzCore, not our code. The render server tells a shown layer's Metal display link which display it is on, on Core Animation's own queue. Below macOS 27, a message handled while the link is being invalidated reads the layer that invalidate() has just cleared. The test shows a layer, changes it and destroys it at once. Cause and evidence (two CI stacks, QuartzCore disassembly, alpine-gpui's identical crash) are in the issue.

The trace layer tests now wait 250 ms before destroying a layer: in TearDown and in the two tests that destroy one midway. The wait narrows the race; it does not remove it, since nothing signals the message.

Verified: the TraceLayer tests (20) in the dev and TSan builds. Locally one dev run had AsksForEachFrameOnTheRenderThreadInTime fail on a stalled display mid-test, before teardown; it passed 3 of 3 on rerun. The race cannot be reproduced on macOS 27, whose QuartzCore checks for the cleared layer. Not run locally: scripts/check.sh and npm run test:app (test-only C++ change).

Considered and not done

  • The product's close path (LayerMac::~LayerMac invalidating the link at once) has the same race on macOS 14 to 26: closing the editor within milliseconds of the render server's display message could crash the host. Moving invalidate() later only narrows it, and a lingering thread risks a host unloading the bundle first. Left for a separate decision.
  • Editor fixtures that create a trace layer through the page (EditorTest, NativeTracesTest, AudioUnitViewTest) share the exposure but show the layer long before teardown and have never crashed. Not changed.

Closes #445

TraceLayerTest.FollowsTheBackingScaleWithNoSizeChange crashed twice on
the macOS 15 runner, once in the merge queue, which ejected the pull
request. The crash is inside QuartzCore: the render server tells a shown
layer's Metal display link which display it is on, on Core Animation's
own queue, and below macOS 27 a message handled while the link is being
invalidated reads the layer that the invalidation has just cleared. The
test shows a layer, changes it and destroys it at once, so the message
can still be in flight when the link is invalidated.

The trace layer tests now wait 250 ms before destroying a layer, in
their teardown and in the two tests that destroy one midway, so the
message is handled while the link still has its layer. This narrows the
race rather than removing it; nothing signals the message. The product's
close path is unchanged.

Verified: the TraceLayer tests pass in the dev and TSan builds. The race
cannot be reproduced on macOS 27, whose QuartzCore checks for the
cleared layer.

Prior art: alpine-gpui PR #644 (same fault, notifyDisplayChange:+152 at
0x20 on CI runners, macOS 26.6.2; "Only shown layers get these
messages"; moving invalidate() away from the show "narrows the race"),
QuartzCore disassembly on macOS 27.2 (MetalLinkItem::invalidate clears
the layer before leaving the link list); Apple's CAMetalDisplayLink
docs list no known issue.
Comment thread editor/tests/TraceLayerTests.mm Resolved
Comment thread editor/tests/TraceLayerTests.mm Resolved
@github-actions

Copy link
Copy Markdown

AI review: no Important findings

PR 446 changes only editor/tests/TraceLayerTests.mm. It adds a destroyLayer() helper that runs the run loop for a fixed 250 ms before resetting a shown TraceLayer, so the CAMetalDisplayLink is not invalidated while a render-server display message is still pending, which crashes on macOS below 27 (#445). The correctness, rules, tests and real-time specialists each reviewed the diff against the pr-head/ files and the base branch's REVIEW.md, CLAUDE.md and .claude/rules.

  • Nit editor/tests/TraceLayerTests.mm:152: Fixed 250 ms sleep (kSettleBeforeDestroying) stands in for a condition and has no stated basis
  • Nit editor/tests/TraceLayerTests.mm:148: The product destroys the trace layer the same way; the change only protects the tests
1 finding(s) not confirmed as Important by verification
  • editor/tests/NativeTracesTests.mm:137: Same teardown crash left unmasked in the other fixtures that destroy a shown TraceLayer. Not confirmed: Verdict: not verified. Severity: nit at most.

Why the finding does not hold:

  1. EditorTests is not affected. The finding says EditorTests runs with native traces on and cites EditorTests.mm:156. That line is TearDownTestSuite, which turns native traces back on after the suite has finished. SetUpTestSuite at pr-head/editor/tests/EditorTests.mm:154 calls EditorCore::setNativeTraces(false), and the comment at lines 150-151 says the page draws the traces in these tests. So none of the roughly 25 editor->removed() calls in EditorTests creates or destroys a TraceLayer. That part of the scenario is false.

  2. The product crash is pre-existing and untouched. The whole diff (ai-review/pr.diff) is in editor/tests/TraceLayerTests.mm. It adds destroyLayer(), which waits 250 ms before layer.reset(), and uses it at three places. No file in editor/src changes. The teardown the finding points to ([state->link invalidate] on the traces thread at editor/src/mac/TraceLayerMac.mm:509, reached from LayerMac::~LayerMac at lines 530-546) is the same before and after this change. The PR does not claim to fix it in the product: the new comment calls it a Core Animation fault below macOS 27, tracked as TraceLayerTest.FollowsTheBackingScaleWithNoSizeChange segfaults intermittently on the macOS runner #445. A host closing the editor soon after opening it is a pre-existing issue outside this change, and the review rules say to reject findings of that kind. "No test fails without a product fix" does not apply, because the change makes no product fix.

  3. What is left is a possible CI flake, which no user meets. NativeTracesTests::close() (pr-head/editor/tests/NativeTracesTests.mm:137-146) does destroy a shown native layer through editor->removed() with no settle wait. So if the new comment's account of the crash is right, that suite could flake the same way. But:

    • Whether a display message is pending at that moment depends on Core Animation timing. Nothing in the code shows it, so I cannot prove a crash on that path.
    • A test flake reaches no user. Under REVIEW.md's severity rules it is not Important.

At most this is a nit: the same 250 ms settle could sit in a shared test helper that NativeTracesTest also uses. Nothing in the code shows a defect that reaches a user and comes from this change.

The author's answers file (ai-review/answers.json) is empty, so no earlier answer covers this finding.

Fix or answer each Important finding. A maintainer can comment /ai-review to review again, or /ai-review waive <reason> to pass the check.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TraceLayerTest.FollowsTheBackingScaleWithNoSizeChange segfaults intermittently on the macOS runner

1 participant