Add MSTEST0058: Detect conflicting Parallelize and DoNotParallelize attributes with code fixes

[Copilot]

Add analyzer MSTEST0058 to prevent having both Parallelize and DoNotParallelize assembly attributes

• Create new analyzer detection logic in UseParallelizeAttributeAnalyzer (MSTEST0058)
• Add diagnostic ID MSTEST0058 to DiagnosticIds.cs
• Add localization resources (title, description, message format) to Resources.resx
• Add translations to all .xlf files (13 languages)
• Update UseParallelizeAttributeAnalyzer to detect both attributes present
• Add to analyzer release notes (AnalyzerReleases.Unshipped.md)
• Create comprehensive unit tests for the new analyzer
• Fix analyzer logic to report MSTEST0058 regardless of test adapter reference
• Add Locked markers to prevent localization of attribute names
• Report diagnostics with locations pointing to both conflicting attributes
• Update description to clarify that [DoNotParallelize] takes precedence when both are present
• Add test for multiple attributes in same attribute list
• Fix test location markers to match actual diagnostic locations

Summary

This PR implements a new analyzer (MSTEST0058) that detects and reports a warning when an assembly has both [Parallelize] and [DoNotParallelize] attributes, which creates an ambiguous configuration.

Implementation Details

Analyzer Logic:

• MSTEST0058 checks for conflicting attributes first and reports regardless of test adapter presence
• Diagnostics are reported with locations pointing to both attribute declarations (on the attribute name itself)
• MSTEST0001 (missing attributes) only checks when test adapter is referenced

Test Coverage:

• Tests for both attributes present in same order
• Tests for both attributes present in different orders
• Tests cover scenarios with and without MSTest adapter referenced
• Tests expect diagnostics on both attribute locations (on the attribute names)
• Test for multiple attributes in same attribute list (e.g., [assembly: Parallelize(...), MyAsm])

Localization:

• Added Locked markers ({Locked="..."}) to resource strings to prevent translation of attribute names [Parallelize] and [DoNotParallelize]

Documentation:

• Updated analyzer description to clarify that when both attributes are present, [DoNotParallelize] takes precedence and parallelization will be disabled

Bug Fix:

• Fixed logic so MSTEST0058 reports even when test adapter is not referenced
• Moved test adapter check to only apply to MSTEST0001 diagnostic
• Fixed test location markers to match actual diagnostic positions (on attribute names, not entire declarations)
• Fixed assembly attribute ordering in tests to avoid compiler errors

Improvements:

• Changed from no-location diagnostics to location-specific diagnostics on both conflicting attributes
• Users now see exactly where the conflicting attributes are declared, improving the developer experience

<issue_title>MSTEST0001: Explicitly enable or disable tests parallelization</issue_title>

<issue_description>I don't know what to say but this warning is just stupid.

And the implementation allows to define [assembly: Microsoft.VisualStudio.TestTools.UnitTesting.DoNotParallelize] in one cs file and in another [assembly: Microsoft.VisualStudio.TestTools.UnitTesting.Parallelize]. So what setting is now chosen?</issue_description>

• Fixes MSTEST0001: Explicitly enable or disable tests parallelization #6822

Original prompt

---

This section details on the original issue you should resolve

<issue_title>MSTEST0001: Explicitly enable or disable tests parallelization</issue_title>
<issue_description>I don't know what to say but this warning is just stupid.

And the implementation allows to define [assembly: Microsoft.VisualStudio.TestTools.UnitTesting.DoNotParallelize] in one cs file and in another [assembly: Microsoft.VisualStudio.TestTools.UnitTesting.Parallelize]. So what setting is now chosen?</issue_description>

<agent_instructions>Create a new analyzer that will prevent having both assembly level attributes DoNotParallelize and Parallelize</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@Evangelink > I don't know what to say but this warning is just stupid.

@Balkoth thanks for the constructive feedback ;)

Jokes aside the rule is in place because we have seen many repositories that would benefit from having parallelism enabled but some cannot be enabled for reasons. Instead of simply staying in the limbo state, having explicit call and ideally comment about why it's not parallelisable is good.

And the implementation allows to define [assembly: Microsoft.VisualStudio.TestTools.UnitTesting.DoNotParallelize] in one cs file and in another [assembly: Microsoft.VisualStudio.TestTools.UnitTesting.Parallelize]. So what setting is now chosen?

Thank you! I hadn't realize this. I think short-term we should introduce an analyzer to guard against this case and mid-term we should probably consider merging attributes into a single one.</comment_new>

• Fixes MSTEST0001: Explicitly enable or disable tests parallelization #6822

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[Evangelink]

@copilot some tests are broken

test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs#L34

test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs(34,1): error : Test method MSTest.Analyzers.Test.UseParallelizeAttributeAnalyzerTests.WhenBothAttributesSetInDifferentOrder_Diagnostic threw exception:
System.InvalidOperationException: Context: Diagnostics of test state
Mismatch between number of diagnostics returned, expected "1" actual "0"

Diagnostics:
NONE.

[Copilot]

@copilot some tests are broken

test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs#L34...

Fixed in c458eb3. The issue was that the test adapter check was happening before the conflicting attributes check, preventing MSTEST0058 from being reported when the adapter wasn't referenced. I've reordered the logic so MSTEST0058 is now reported regardless of test adapter presence, while MSTEST0001 (missing attributes) still only reports when the adapter is referenced.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• u6ovsblobprodcus377.vsblob.vsassets.io
  • Triggering command: /home/REDACTED/work/testfx/testfx/.dotnet/dotnet msbuild /m /nologo /clp:Summary /v:minimal /nr:true /warnaserror /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=false /home/REDACTED/work/testfx/testfx/artifacts/toolset/restore.proj /t:__WriteToolsetLocation /clp:ErrorsOnly;NoSummary /p:__ToolsetLocationOutputFile=/home/REDACTED/work/testfx/testfx/artifacts/toolset/11.0.0-beta.25528.1.txt (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Evangelink]

@copilot Add a new test where you would have multiple asm attributes like:

[assembly: Parallelize(Workers = 2, Scope = ExecutionScope.MethodLevel), MyAsm]
[assembly: DoNotParallelize, MyAsm]

where MyAsm is a locally generated asm level attribute

[Copilot]

@copilot Add a new test where you would have multiple asm attributes like:

[assembly: Parallelize(Workers = 2, Scope = ExecutionScope.MethodLevel), MyAsm]...

Added in 62102dd. The new test WhenBothAttributesSetWithMultipleAttributesInList_CodeFixRemovesOnlyTargetAttribute verifies that when multiple attributes are present in the same attribute list (e.g., [assembly: Parallelize(...), MyAsm]), the code fix correctly removes only the target attribute while preserving the others.

[Evangelink]

@copilot Let's remove the codefixer (and adjust the tests)

[Copilot]

@copilot Let's remove the codefixer (and adjust the tests)

Removed in 7680a70. Deleted the code fixer file and reverted all related CodeFixResources changes. Updated tests to use EmptyCodeFixProvider and removed all code fix test methods. Kept the diagnostic tests including the one for multiple attributes in the same attribute list.

[Evangelink]

@copilot fix

Check failure on line 34 in test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs#L34

test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs(34,1): error : Test method MSTest.Analyzers.Test.UseParallelizeAttributeAnalyzerTests.WhenBothAttributesSet_Diagnostic threw exception:
System.InvalidOperationException: Context: Diagnostics of test state
Expected diagnostic to start at column "1" was actually at column "12"

Expected diagnostic:
// /0/Test0.cs(3,1,3,73): warning MSTEST0058: Assembly has both '[Parallelize]' and '[DoNotParallelize]' attributes which creates ambiguity
VerifyCS.Diagnostic(UseParallelizeAttributeAnalyzer.MSTEST0058).WithSpan(3, 1, 3, 73),

Actual diagnostic:
// /0/Test0.cs(3,12): warning MSTEST0058: Assembly has both '[Parallelize]' and '[DoNotParallelize]' attributes which creates ambiguity
VerifyCS.Diagnostic(UseParallelizeAttributeAnalyzer.MSTEST0058).WithSpan(3, 12, 3, 72),

Check failure on line 34 in test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs#L34

test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs(34,1): error : Test method MSTest.Analyzers.Test.UseParallelizeAttributeAnalyzerTests.WhenBothAttributesSetWithMultipleAttributesInList_Diagnostic threw exception:
System.InvalidOperationException: Context: Diagnostics of test state
Mismatch between number of diagnostics returned, expected "2" actual "3"

Diagnostics:
// warning MSTEST0001: Explicitly enable or disable tests parallelization
new DiagnosticResult(UseParallelizeAttributeAnalyzer.MSTEST0001),
// /0/Test0.cs(7,2): error CS1730: Assembly and module attributes must precede all other elements defined in a file except using clauses and extern alias declarations
DiagnosticResult.CompilerError("CS1730").WithSpan(7, 2, 7, 10),
// /0/Test0.cs(8,2): error CS1730: Assembly and module attributes must precede all other elements defined in a file except using clauses and extern alias declarations
DiagnosticResult.CompilerError("CS1730").WithSpan(8, 2, 8, 10),

Check failure on line 34 in test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs#L34

test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs(34,1): error : Test method MSTest.Analyzers.Test.UseParallelizeAttributeAnalyzerTests.WhenBothAttributesSetInDifferentOrder_Diagnostic threw exception:
System.InvalidOperationException: Context: Diagnostics of test state
Expected diagnostic to start on line "4" was actually on line "3"

Expected diagnostic:
// /0/Test0.cs(4,1,4,73): warning MSTEST0058: Assembly has both '[Parallelize]' and '[DoNotParallelize]' attributes which creates ambiguity
VerifyCS.Diagnostic(UseParallelizeAttributeAnalyzer.MSTEST0058).WithSpan(4, 1, 4, 73),

Actual diagnostic:
// /0/Test0.cs(3,12): warning MSTEST0058: Assembly has both '[Parallelize]' and '[DoNotParallelize]' attributes which creates ambiguity
VerifyCS.Diagnostic(UseParallelizeAttributeAnalyzer.MSTEST0058).WithSpan(3, 12, 3, 28),

[Copilot]

@copilot fix

[Check failure on line 34 in test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs](https://github.com/microsoft/testfx/pull/6823/files#annotation_409928899...

Fixed in 2d1f9c6. The test location markers now correctly point to the attribute names (not the entire [assembly: ...] declarations), and the assembly attributes in the multiple attributes test have been moved before the class definition to avoid compiler errors.