Skip to content

fix(macos): invalidate the display link only after the window close - #644

Merged
dbuddha merged 4 commits into
mainfrom
fix/533-invalidate-after-close
Oct 2, 2026
Merged

dbuddha merged 4 commits into
mainfrom
fix/533-invalidate-after-close

Conversation

@dbuddha

@dbuddha dbuddha commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Context

#533: native_wake crashes with SIGSEGV on hosted runners, about 1 in 800 runs: 3 of 2,400 in the same-VM A/B (run 37015643917), at the same rate on main and with M0e.1. Core dumps from that run gave the first stacks. All three show:

  • QuartzCore +[CADisplayLink(CADisplayLinkInternal) notifyDisplayChange:] + 152 faulting at address 0x20 on a dispatch worker thread (invoke_display_change_callback);
  • the main thread inside -[NSWindow close], after our windowWillClose handler had invalidated the CAMetalDisplayLink.

Root cause

From disassembling QuartzCore on macOS 26.6.2 (done in the independent review):

  • notifyDisplayChange:+152 loads the layer pointer of a CA::Display::MetalLinkItem, then its CAMetalLayer._priv ivar at offset 0x20.
  • That callback is dispatched from the render server's ImageQueueDidChangeDisplay message to a block that -[CAMetalLayer init] installs. The block finds our item in MetalLinkItem::_list and reads it without a lock.
  • MetalLinkItem::invalidate() both nulls the layer pointer and removes the item from that list.

So the crash is a true race: an invalidate() on the main thread overlaps a callback that has already passed its list lookup. Main invalidated inside windowWillClose, at the start of a close and shortly after the window was shown. Only shown layers get these messages, which fits: the revoked surface, never shown, never crashed.

This PR moves the single invalidate() away from the close, to owner teardown. It narrows the race; it does not make QuartzCore's callback safe. A late callback at teardown could still overlap, and the A/B measures whether it does.

Change

  • At close, retire_display_link pauses the link and clears its delegate, but does not invalidate it. Later directives are ignored, so a retired link never resumes.
  • The single invalidate() happens at owner teardown (NativeSurface Drop and the builder's failure path), after the window close has returned and just before the fields release the link.
  • A new validation probe counter, early_link_invalidations, counts invalidations made before teardown marks the close finished. Tests assert:
    • 0 invalidations at close;
    • exactly 1 after teardown;
    • 0 early.
  • post-commit-close also asserts that no display-link callback reaches the delegate in the 100 ms after close.

Evidence

  • Before: with the new probe and the base close behaviour, native_wake panicked right after windowWillClose (link_invalidations: 1, early_link_invalidations: 1). The silent-close child and native_initialization failed the same way.
  • After: all of them pass.
  • Same-VM hosted A/B (run 37032035015): four VMs alternated main fd2a390 and this fix, 5,000 native_wake runs per arm.
    • SIGSEGV: 3 of 5,000 on main, 0 of 5,000 with the fix.
    • Missing-close failures: 0 of 400 on both.
    • Taken alone, 3 vs 0 is weak (p about 0.125). Against the pooled earlier rate (8 crashes in about 8,700 pre-fix runs), 0 in 5,000 has about a 1% chance.
    • Second batch (run 37035759936): main 3 SIGSEGV plus 2 drain timeouts in 5,000; the fix 0 failures of any kind in 5,000.
    • Combined: 6 of 10,000 on main, 0 of 10,000 with the fix. If the arms were equal, all six crashes would land on main by chance 1.6% of the time.
  • The lead gate on 1760a87 is posted as a comment.

Not covered

  • Drop closing the window itself. When a surface is dropped without an earlier close, Drop invalidates right after close() returns. QuartzCore's display-change callback is asynchronous, so the race may survive on that path. The user-close and quit paths close the window before Drop.
  • Closed links stay registered. A closed link stays on the main run loop, paused and with no delegate, until the surface is dropped.
  • Probe timing. The probe's "close finished" point is owner teardown, not an independent AppKit signal. early_link_invalidations counts invalidations made outside teardown; it is a regression guard and fails on base.
  • Residual race. QuartzCore's callback stays unsynchronized. A callback that arrives exactly at teardown could still overlap the single invalidate().

Risk and scope

This changes the product close path in native.rs (+83/-19, including the probe). No API changes, no new dependencies, and no relaxed invariant: one invalidation per surface, and no callbacks after close. crates/AGENTS.md gains the lifecycle rule, and the #533 known defect is updated from the A/B result.

Test plan

🤖 Generated with Claude Code

…inishes

QuartzCore can walk an invalidated but unreleased link while a window orders
out, so a close only pauses and detaches it and teardown invalidates it.
invalidate() races QuartzCore's display-change callback; it is not a walk
of an invalidated link. Adds the lifecycle rule to crates/AGENTS.md.
Copilot AI balanced review requested due to automatic review settings October 2, 2026 16:43
@dbuddha

dbuddha commented Oct 2, 2026

Copy link
Copy Markdown
Owner Author

Lead gate on 1760a87, the code head; later commits are comments and docs only:

  • fmt and clippy -D warnings clean;
  • scripts/check.sh passes;
  • scripts/check-native.sh hosted all passes, including the silent display link stage;
  • the known-flaky script passes (native_wake 0 of 25).

Root next: gets updated after #643 merges, to avoid a conflict on the same line.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@gemini-code-assist-alpine-gpui gemini-code-assist-alpine-gpui Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses a race condition (SIGSEGV, #533) where invalidating a display link during a window close raced with QuartzCore's display-change callback. The fix introduces a retire_display_link step that pauses and detaches the link during the close, deferring the actual invalidate() call to owner teardown. Additionally, new validation metrics (early_link_invalidations and callback_observations) and corresponding test assertions have been added to ensure display links are not invalidated prematurely. There are no review comments, so no further feedback is provided.

Copilot AI balanced review requested due to automatic review settings October 2, 2026 17:13
@dbuddha

dbuddha commented Oct 2, 2026

Copy link
Copy Markdown
Owner Author

Independent review handled.

  • No blockers. Lifecycle verified: no path resumes a retired link, Drop order is correct, no callbacks after close, and no earlier guarantee is loosened.
  • Should-fix 1, wrong mechanism: corrected in 91bf945, in the comments, the probe doc and the PR body. Per the reviewer's disassembly it is a race between invalidate() and QuartzCore's display-change callback, not a walk of an invalidated link.
  • Should-fix 2, the race may only move: measured instead of argued. Two same-VM batches gave main 6 SIGSEGV in 10,000 and this fix 0 in 10,000, with no teardown crashes. The residual overlap at teardown and the Drop-closes-window path are listed under Not covered. I did not run the suggested hosted lldb trace; the outcome data decided this.
  • Nit, the probe is true by construction: the doc now says it counts invalidations outside owner teardown. It stays as a regression guard and fails on base.

Main (with #643) is merged in, and the lead gate passes on 3d19e64. next: closes M0e.2. Arming auto-merge.

@dbuddha
dbuddha enabled auto-merge (squash) October 2, 2026 17:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@dbuddha
dbuddha merged commit 26597c0 into main Oct 2, 2026
5 checks passed
@dbuddha
dbuddha deleted the fix/533-invalidate-after-close branch October 2, 2026 17:30
dbuddha added a commit that referenced this pull request Oct 2, 2026
* ci: require the native_wake and native_process runs again

Both pass on main after #641 and #644, so they move back into the required
metal-validation job and the known-flaky job and script are deleted.

* docs: correct the .ips claim, keep crash reports, start M1

Hosted runners do write .ips reports; metal-validation now uploads them on
failure. M1 is active with its four slices in next: (owner approved).

* docs: say the 2.1 cold path, not cold start
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants