[NativeAOT] Crash during gen1 GC

[tomcashman]

I've been contributing to NativeAOT port on game consoles by @TheSpydog . With our current project, we are receiving a crash during gen1 GC using .NET 7 preview 7 on an ARM64 platform.

	Bootstrap.nss!WKS::WaitLonger(int i) Line 1490 (0x33B9216E00)	C++
 	Bootstrap.nss!WKS::gc_heap::background_mark_phase() Line 34085 (0x33B922AB10)	C++
 	Bootstrap.nss!WKS::gc_heap::gc1() Line 20663 (0x33B9229830)	C++
 	Bootstrap.nss!WKS::gc_heap::bgc_thread_function() Line 35184 (0x33B92402F8)	C++

We're unsure if this is an upstream issue or perhaps a misconfiguration of NativeAOT on our side. I've included the info that I was able to find below. Any additional pointers to help diagnose further would be much appreciated.

IsCurrentThreadInCooperativeMode is returning false, triggering the assert in WaitLonger(int i).

bool Thread::IsCurrentThreadInCooperativeMode()
{
#ifndef DACCESS_COMPILE
    ASSERT(ThreadStore::GetCurrentThread() == this);
#endif // !DACCESS_COMPILE
    return (m_pTransitionFrame == NULL);
}

I think the issue is related to volatile read/write of m_pTransitionFrame in thread.cpp.

void Thread::EnablePreemptiveMode()
{
    ASSERT(ThreadStore::GetCurrentThread() == this);
#if !defined(HOST_WASM)
    ASSERT(m_pDeferredTransitionFrame != NULL);
#endif

    Unhijack();

    // set preemptive mode
    VolatileStoreWithoutBarrier(&m_pTransitionFrame, m_pDeferredTransitionFrame);
}

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[EgorBo]

Could be related to #72645 ? (with a fix in #74325)

[VSadov]

bgc_thread_function - I do not think background GC works on NativeAOT yet.

(cc @MichalStrehovsky, @jkotas is that true? I do not see write watch support in Windows-arm64 barriers, so I assume background GC is not something that we support/test yet)

Regardless, for an initial port, I'd suggest not enabling "extras" such as background GC or card bundles.

[tomcashman]

At the moment we're using software write watch.

These are all the preprocessor definitions we're using. Perhaps we need to add/remove some?

TARGET_UNIX;TARGET_ARM64;_ARM64_;_aarch64_;HOST_ARM64;HOST_64BIT;_LIBUNWIND_DISABLE_ZERO_COST_APIS=1;_LIBUNWIND_IS_NATIVE_ONLY;_LIBUNWIND_HAS_NO_THREADS;_LIBUNWIND_USE_DLADDR=0;_LIBUNWIND_IS_BAREMETAL;_LIBUNWIND_USE_ONLY_DWARF_INDEX;PAL_STDCPP_COMPAT;FEATURE_BASICFREEZE;FEATURE_USE_SOFTWARE_WRITE_WATCH_FOR_GC_HEAP;FEATURE_CONSERVATIVE_GC;FEATURE_CUSTOM_IMPORTS;FEATURE_NATIVEAOT;FEATURE_FIXED_POOL_THUNKS;VERIFY_HEAP;NATIVEAOT;FEATURE_CACHED_INTERFACE_DISPATCH;_LIB;NO_UI_ASSERT;FEATURE_GC_STRESS;

[jkotas]

I do not think background GC works on NativeAOT yet.

It should work when software write watch is available (ie Windows). It does not work when software write watch is not available.

I do not see write watch support in Windows-arm64 barriers

We should fix that (it is missing on all platforms) but it unlikely to be be the root cause of this crash.

[VSadov]

@tomcashman the easiest way to disable background GC is by configuring GC (https://docs.microsoft.com/en-us/dotnet/core/runtime-config/garbage-collector). The knob that controls background GC is gcConcurrent

i.e.
export COMPlus_gcConcurrent=0

Even if it is supposed to work, it may be a good idea to disable it during initial bring up.

[tomcashman]

It should work when software write watch is available (ie Windows). It does not work when software write watch is not available.

@jkotas is software write watch not supported on Linux? When we originally looked through the code we thought it would work.

@VSadov I'll test without background GC but not ideal as we're porting video games which can't really pause on the main thread without killing frame rate.

[jkotas]

@jkotas is software write watch not supported on Linux? When we originally looked through the code we thought it would work.

We are missing a bit of assembly code required to make it work.

[tomcashman]

We are missing a bit of assembly code required to make it work.

This section of src\coreclr\nativeaot\Runtime\gcrhenv.cpp:1171 stated that only UNIX/ARM64 is supported for software write watch.

#if defined(FEATURE_USE_SOFTWARE_WRITE_WATCH_FOR_GC_HEAP) && (!defined(TARGET_ARM64) || !defined(TARGET_UNIX))
#error FEATURE_USE_SOFTWARE_WRITE_WATCH_FOR_GC_HEAP is only implemented for ARM64 and UNIX
#endif

That was what led us to enabling it for background GC. Now we're confused 😄

[VSadov]

@jkotas - yes the crash is because background GC threads are expected to be in cooperative mode. Somehow that is not happening in @tomcashman`s scenario, but it can't be a race since this is an assert from the same thread. It is a separate issue.

I just wanted to be sure we do expect background GC to run. I have never seen background threads while debugging or looking at the dumps so I assumed we might not be enabling BGC. BGC threads are created and destroyed on demand though, so I might just never seen them in action.

I also noticed that some barriers are missing the parts that update the write watch (I think it is the Windows that is missing these parts, but Unix has them, at least on arm64). It is possible that Windows may be using hardware writewatch, which does not rely on barriers.

[VSadov]

I'll test without background GC but not ideal as we're porting video games which can't really pause on the main thread without killing frame rate.

@tomcashman Let me dig a bit further whether BGC should/can/does work.
it looks like the support for background GC is there, at least on Unix+arm64, so it might work. It is strange that we do not see the same assert in tests though.

[tomcashman]

If there's anything I can do to help diagnose, please let me know.