QUIC failures on linux

[AndyAyersMS]

I have seen various QUIC failures on arm64 linux in different PRs and in different pipelines. Symptoms vary, but here is a crash.

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=187390
Build error leg or test failing: System.Net.Quic.Functional.Tests.WorkItemExecution
Pull request: #82752

Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "Segmentation fault      (core dumped) \"$RUNTIME_PATH/dotnet\" exec --runtimeconfig System.Net.Quic.Functional.Tests",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Report

Build	Definition	Test	Pull Request
225373	dotnet/runtime	System.Net.Quic.Functional.Tests.WorkItemExecution	#84207
220949	dotnet/runtime	System.Net.Quic.Functional.Tests.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	2

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I have seen various QUIC failures on arm64 linux in different PRs and in different pipelines. Symptoms vary, but here is a crash.

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=187390
Build error leg or test failing: System.Net.Quic.Functional.Tests.WorkItemExecution
Pull request: #82752

Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "Segmentation fault      (core dumped) \"$RUNTIME_PATH/dotnet\" exec --runtimeconfig System.Net.Quic.Functional.Tests",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Author:	AndyAyersMS
Assignees:	-
Labels:	blocking-clean-ci, untriaged, area-System.Net.Quic, Known Build Error
Milestone:	-

[wfurt]

https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-82752-merge-e1a10947e940484c88/System.Net.Quic.Functional.Tests/1/console.d04fe702.log?helixlogtype=result

claims to be amd64. And it seems like there is no crash dump so it will be hard to see what is happening.

[AndyAyersMS]

Ooops yes that one is amd64. There is a crash dump though.

https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-82752-merge-e1a10947e940484c88/System.Net.Quic.Functional.Tests/1/core.1000.14745?helixlogtype=result

[AndyAyersMS]

This is another one, probably different issue as it's a time out: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-heads-main-919e7e57ce2b4e4ab5/System.Net.Quic.Functional.Tests/1/console.03606f93.log?helixlogtype=result

[wfurt]

(lldb) bt
* thread #1, name = 'dotnet', stop reason = signal SIGSEGV
  * frame #0: 0x00007eff8a02c85b libmsquic.so.2`CxPlatRefDecrement at platform_posix.c:339:5
    frame #1: 0x00007eff8a02c83c libmsquic.so.2`CxPlatRefDecrement(RefCount=0x0000001100000124) at platform_posix.c:325
    frame #2: 0x00007ebe44073ee0

It does seems like some MsQuic or Quic issue.

[wfurt]

and msquic version

(lldb) dumpobj 00007ebefe0c9368
Name:        System.String
MethodTable: 00007eff13f5aed8
EEClass:     00007eff13fe0760
Size:        184(0xb8) bytes
File:        /datadisks/disk1/work/A2890920/p/shared/Microsoft.NETCore.App/8.0.0/System.Private.CoreLib.dll
String:      libmsquic.so version=2.1.7.329711 commit=2db78a2cd72d0c95111d5f4884fbc17307a4238d

so this is not the updated msquic @ManickaP