Fix flaky tests by locking on the statup type

[remcolam]

@martincostello I think this will contribute to #2836

[martincostello]

Thanks - I'll run the build a few times and if it passes every time I'll merge it 🙇‍♂️

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.06%. Comparing base (4baef30) to head (b26b6a3).
Report is 4 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2838      +/-   ##
==========================================
- Coverage   92.13%   92.06%   -0.08%     
==========================================
  Files          91       93       +2     
  Lines        3088     3111      +23     
  Branches      529      536       +7     
==========================================
+ Hits         2845     2864      +19     
- Misses        243      247       +4     

Flag	Coverage Δ
Linux	92.06% <ø> (-0.08%)	⬇️
Windows	92.06% <ø> (-0.08%)	⬇️
macOS	92.06% <ø> (-0.08%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[remcolam]

@martincostello Actually, thinking about it, I think the lock needs to be larger, encapsulating the whole test. Because while the client is running, you are not able to compile it.

[martincostello]

There's also a failure after the tests end where there's a write conflict on the coverage report:

2024-04-25T10:44:39: Error during rendering summary report (Report type: 'Html'): The process cannot access the file 'D:\a\Swashbuckle.AspNetCore\Swashbuckle.AspNetCore\artifacts\coverage\index.html' because it is being used by another process.
  Basic -> D:\a\Swashbuckle.AspNetCore\Swashbuckle.AspNetCore\artifacts\bin\Basic\release_netcoreapp3.1\Basic.dll
  Out-File: The process cannot access the file 'D:\a\_temp\_runner_file_commands\step_summary_23f9f5a4-591b-4ba4-ac37-76846145df0c' because it is being used by another process.
D:\a\Swashbuckle.AspNetCore\Swashbuckle.AspNetCore\Directory.Build.targets(25,5): error MSB3073: The command "pwsh -Command "('<details><summary>:chart_with_upwards_trend: <b>Swashbuckle.AspNetCore.Annotations.Test Code Coverage report</b></summary>' + [System.Environment]::NewLine + [System.Environment]::NewLine + (Get-Content D:\a\Swashbuckle.AspNetCore\Swashbuckle.AspNetCore\artifacts\coverage\SummaryGithub.md | Out-String) + [System.Environment]::NewLine + [System.Environment]::NewLine + '</details>') >> D:\a\_temp\_runner_file_commands\step_summary_23f9f5a4-591b-4ba4-ac37-76846145df0c"" exited with code 1. [D:\a\Swashbuckle.AspNetCore\Swashbuckle.AspNetCore\test\Swashbuckle.AspNetCore.Annotations.Test\Swashbuckle.AspNetCore.Annotations.Test.csproj]
  Swashbuckle.AspNetCore.SwaggerGen.Test -> D:\a\Swashbuckle.AspNetCore\Swashbuckle.AspNetCore\artifacts\bin\Swashbuckle.AspNetCore.SwaggerGen.Test\release_net6.0\Swashbuckle.AspNetCore.SwaggerGen.Test.dll

[remcolam]

I already realized it won't work... the lock is not long enough and putting the whole method in it will not allow await

[remcolam]

This reworked version has run 50 times in parallel for me (however, I was only running the integratin tests, not other tests)

[martincostello]

whole method in it will not allow await

Await is possible within lock in recent versions of C# IIRC.

[remcolam]

Noth that I could find, MS documentation also states it is not possible: https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/lock?devlangs=csharp&f1url=%3FappId%3DDev17IDEF1%26l%3DEN-US%26k%3Dk(lock_CSharpKeyword)%3Bk(SolutionItemsProject)%3Bk(DevLang-csharp)%26rd%3Dtrue

[martincostello]

Maybe I've confused it with await support for finally

[remcolam]

@martincostello I was reading up on xunit, tests inside a class are run sequentially, so I think the issue is actually that the client wasn't being disposed, so depending on the garbage collector running you would run into the issue. I added a using to all the methods

[martincostello]

Ah cool - I had noticed that myself in a few tests when doing work on them, but I hadn't gone through all the specific usages to ensure everything is disposed of.

[remcolam]

Hopefully this now fixes the locking issues... I haven't run into it, hopefully now it solves it for you as well

[martincostello]

I'll run the CI a few times and see how it goes. I've updated the PR description to not close the associated issue as I don't think this change will fix the case where it fails due to conflicts when writing the code coverage report(s): #2838 (comment)

[martincostello]

7 retries and no failures 👍

[remcolam]

Happy to help

[remcolam]

I think you could call me doing this in dutch Woggen (tranlates to english as Wabbing). Making a verb from the acronym for Werk Ontwijkend Gedrag/Work Avoiding Behavior)